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Abstract 

The  haptic  interaction  of  humans  with  soft  objects  was  studied  from  three  perspectives; 
softness  discrimination,  force  control,  and  contact  visualization.  The  abilities  of  humans  in 
actively  discriminating  softness  was  measured  by  using  a  specimen  presenter  system  which 
was  built  to  randomly  |)rcsent  the  specimens.  Two  experimental  paradigms  (1-  and  2-finger) 
and  three  finger  conditions  (normal,  finger  cot,  and  rigid  thimble)  were  used  to  examine 
the  importance  of  various  sources  of  information.  When  using  1  finger,  the  .Just  Noticeable 
Difference  (JND)  was  about  5%  for  normal  and  finger  cot  conditions  and  increased  to  about 
•50%  with  the  thimble.  The  .JND  results  from  2-finger  discrimination  were  lower  than  the 
1-finger  results  for  both  normal  and  finger  cot  conditions,  but  were  higher  with  the  thimble. 
Examination  of  the  forces  exerted  on  the  specimens  during  2-finger  discriminations  revealed 
possible  underlying  discrimination  strategies.  In  the  force  control  experiments,  subjects  were 
asked  to  exert  several  levels  of  constant  forces  under  various  finger  conditions.  The  results 
indicated  that  the  errors  from  tracking  with  visual  feedback  was  significantly  lower  than 
that  without  visual  feedback.  No  significant  differences  in  force  control  were  found  with 
either  different  softnesses  or  under  the  three  finger  conditions.  The  absolute  errors  were 
higher  when  controlling  higher  target  forces.  Significant  difference  in  force  control  was  found 
between  the  tv.’o  hands  of  the  subject  who  showed  handedness  in  the  softness  discrimination 
experiments.  For  contact  visualization,  a  real-time  imaging  setup  was  built  which  consisted 
of  a  videomicroscopy  system  and  a  tactile  stimulator  system.  By  using  this  setup,  real-time 
images  from  the  contact  region  as  /Well  as  the  contact  forces  were  digitized.  Various  image 
processing  techniques  were  developed  and  applied  in  order  to  analyze  and  improve  the  contact 
images  to  distinguish  between  the  contact  and  non-contact  regions.  Contact  variables  such 
as  force,  nominal  contact  area,  actual  contact  area,  nominal  pressure  and  actual  pressure 
were  analyzed.  Based  on  the  visualization  of  the  active  slip  phenomenon,  a  hypothesis  on 
the  contact  pressure  distribution  was  proposed.  The  developed  hardware  can  be  used  for 
evaluating  human  haptic  abilities  and  providing  biomechanical  visualizations.  The  results 
obtained  have  implications  on  the  mechanisms  of  contact  and  find  applications  in  dextrous 
robot  finger  and  haptic  interface  designs  as  well  as  in  various  steps  involved  in  the  automatic 
fingerprint  identification  systems. 

Thesis  Supervisor:  Dr.  Mandayam  A.  Srinivasan 
Title:  Principal  Research  Scientist 
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Chapter  1 


Introduction 

1.1  Human  haptic  interaction  with  soft  objects 

Our  hands  are  a,  part  of  the  human  haptic  system  which  is  comprised  of  the  biomechanical, 
sensoimotor  and  cognitive  subsystems.  Complete  understanding  of  the  human  haptic  system 
requires  an  analysis  of  the  mechanics  of  contact  together  with  other  aspects  of  the  system  such 
as  physiology,  neuroscience  and  psychophysics.  When  we  interact  with  objects  in  an  environ¬ 
ment,  useful  information  is  gathered  from  information  sources  involving  visual,  auditory,  and 
haptic  systems.  Exploration  or  manipulation  of  objects  generally  involves  multimodal  per¬ 
ception  and  intersensory  integration  [15][46][48].  Even  though  the  interaction  is  multimodal, 
we  rely  heavily  on  our  hands  to  perceive  object  properties  and  control  the  interaction.  This  is 
because  the  tactile  information  originates  directly  from  the  finger-object  contact  interface  as 
a  result  of  contact  forces  imposed  on  the  fingerpads.  The  mechanoreceptors  underneath  the 
skin  respond  to  the  imposed  loading  and  the  generated  neural  signals  are  transmitted  to  the 
brain  through  pathways  of  the  central  nervous  system.  To  understand  what  information  is 
available  to  the  sensors  underneath  the  skin,  visualizing  the  contact  process  is  an  important 
step.  In  this  stud}',  therefore,  a  videomicroscopy  system  was  built  to  visualize  the  contact  re¬ 
gion  of  the  fingerpad  with  transparent  objects.  The  image  sequences  obtained  from  fingerpad 
contact  w'ith  rigid  Plexiglas  and  transparent  objects  of  different  softnesses  were  observed  and 
analyzed  by  applying  various  image  processing  techniques  to  extract  contact  information. 

At  a  behavioral  level,  human  perception  of  the  object  properties  related  to  the  human 
haptic  system  have  been  studied  in  terms  of  the  detection,  discrimination,  and  recognition 
(see  Sherrick  &  Cholewuak  [37]  and  Loomis  &  Lederman  [38]  for  a  review).  The  stimuli  that 
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have  been  used  cover  a  wide  variety  such  as  point  loads,  surface  irregularities,  vibrotactile 
threshold,  patterns,  roughness,  and  texture.  However,  not  much  is  known  about  the  human 
discriminability  of  rubber-like  soft  objects.  Early  studies  on  softness  perception  existed 
in  forms  of  stimuli  ranking,  scaling,  and  psychoplij^sical  laws:  for  example,  studies  on  the 
skills  of  test  bakers[27],  cheese  maker  and  grader  craftsmanship,  and  on  scaling,  matching 
and  cross-modal  comparisons  of  softness  perception[20].  One  severe  problem  that  hinders 
quantitative  understanding  in  such  studies  is  the  lack  of  of  high  quality  stimuli  which  differ 
only  in  softness  [28].  In  this  stud,y,  human  softness  discrimination  ability  was  examined  by 
performing  discrimination  experiments  with  visually  identical,  smooth,  compliant  objects  that 
had  no  perceivable  surface  features  and  differed  only  in  softness  which  could  be  controlled 
and  characterized  to  a  high  level  of  precision. 

Two  sources  of  information  are  available  during  haptic  interaction  with  objects.  Tactile 
information  is  gathered  from  the  sensors  embedded  in  the  skin  during  contact;  kinesthetic 
information  is  generated  by  the  sensory  receptors  around  joints,  tendons  and  muscles  [11] [21]. 
In  order  to  determine  the  relative  importance  of  tactile  information  at  both  gross  and  detailed 
level  as  well  as  that  of  kinesthetic  information  during  softness  discrimination,  three  different 
finger  conditions  were  examined:  normal  finger,  finger  covered  with  a  finger  cot  and  with  a 
rigid  thimble.  With  normal  finger  and  finger  covered  with  a  thin  glove-like  latex  finger  cot. 
both  tactile  information  and  kinesthetic  information  are  available.  The  presence  of  finger  cot 
can,  however,  be  thought  of  as  a  thin  layer  on  top  of  the  finger  ridges  which  would  remove 
some  of  the  fine  details  that  the  finger  would  otherwise  perceive  with  normal  fingers.  The 
rigid  thimble  replaces  the  fingerpad-object  interface  with  thimble-object  and  thimble-finger 
interfaces.  Thus,  when  discriminating  softness  with  the  thimble  on,  useful  tactile  information 
which  would  otherwise  be  obtained  from  the  contact  region  are  removed.  Therefore,  the 
subjects  would  need  to  rely  on  the  unperturbed  kinesthetic  information. 

In  an  attempt  to  address' multifinger  interaction  with  objects,  an  experimental  paradigm 
that  used  the  two  index  fingers  from  both  hands  was  also  tested  in  addition  to  one-finger  soft¬ 
ness  discrimination  experiments.  Aside  from  analyzing  the  Just  Noticeable  Difference  (JND) 
results,  the  forces  applied  for  discrimination  were  also  examined  to  reveal  the  strategies  ad¬ 
opted  by  the  subjects.  To  further  understand  the  underlying  limitations  in  the  ability  to 
achieve  fine  discriminations  of  softness,  force  control  experiments  were  conducted  to  examine 
the  subjects’  abilities  in  controlling  normal  forces  of  contact  with  objects  of  different  soft- 
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nesses.  Both  left  and  right  index  fingers  under  normal,  finger  cot,  and  thimble  conditions 
were  examined  in  following  and  controlling  three  different  constant  target  force  magnitudes 
with  and  without  visual  force  feedback  available. 

1.2  Specific  aims  of  the  thesis 

The  specific  aims  of  this  thesis  are  (1)  to  develop  a  videomicroscop\'  system  for  visualization 
of  the  contact  region  of  the  human  fingerpad  during  contact  with  transparent  objects,  (2) 
to  develop  a  single  degree  of  freedom  tactile  stimulator  capable  of  transporting  stimulus 
objects  to  indent  tlie  fingerpad  at  a  controlled  velocity,  (3)  to  develop  a  specimen  presenter 
to  perform  psychophysical  experiments  on  the  human  haptic  discrimination  of  object  softness, 
(4)  to  characterize  the  human  ability  in  controlling  normal  finger  forces  with  specimens  of 
vai  ious  soft  nesses  under  several  contact  interface  conditions,  (5)  to  perform  passive  and  active 
touch  experiments  involving  the  human  finger  and  transparent  soft  specimens  to  characterize 
how  the  contact  region  is  affected  by  the  parameters  such  as  contact  force  and  specimen 
compliance,  and  (6)  to  gain  a  deeper  understanding  of  how  contact  information  and  force 
control  affect  the  ability  to  discriminate  softness.  It  is  hoped  that  the  results  of  this  study  on 
the  human  haptic  performance  and  the  underlying  mechanisms  will  also  provide  useful  data 
and  insights  for  the  design  of  haptic  interfaces  and  dextrous  robot  fingers. 

1.3  Organization  of  the  thesis 

This  thesis  is  organized  as  three  parts;  softness  discrimination,  force  control,  and  con¬ 
tact  visualization.  Background,  devices,  experiments,  and  results  related  to  each  topic  are 
provided  in  the  corresponding  chapters. 

Chapter  2  describes  the  measurement  of  the  human  ability  in  softness  discrimination. 
Background  information  on  the  human  haptic  system  and  previous  work  on  softness  discrim¬ 
ination  are  provided.  A  device  for  specimen  presentation  was  designed  and  built  to  measure 
the  human  ability  in  softness  discrimination  under  various  finger  conditions  and  experimental 
paradigms.  The  results  are  discussed  and  the  forces  used  by  subjects  are  examined. 

In  Chapter  3,  the  measurement  of  the  human  ability  in  normal  force  control  with  soft 
objects  is  presented.  Background  information  on  the  related  human  sensory  abilities  are 
examined.  Experiments  to  determine  the  human  abilities  in  controlling  normal  forces  of 
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contact  with  objects  of  various  softnesses  and  under  various  experimental  conditions  were 
conducted.  The  performance  of  the  subjects  on  force  control  under  various  conditions  were 
measured  and  compared. 

The  efforts  on  contact  visualization  is  presented  in  Chapter  4.  A  videomicroscopy  system 
was  built  to  enable  visualization  of  the  contact  region  of  the  human  finger  with  transparent 
objects.  .4  tactile  stimulator  system  was  built  to  conduct  controlled  contact  experiments. 
With  the  integration  of  the  two  systems,  real-time  experiments  to  observe  both  contact  region 
and  contact  forces  are  now  possible.  The  device  was  used  to  acquire  images  under  both  active 
and  passive  touch  experiments  to  analyze  contact  information.  Various  image  processing 
techniques  were  applied  to  facilitate  phenomenological  descriiJtion  and  to  quantify  the  image 
data,  for  overall  (nominal)  and  actual  contact  areas. 

Chapter  5  concludes  the  main  results  from  the  three  aspects  of  softness  discrimination 
detailed  in  the  previous  chapters.  Implications  of  the  results  on  human  haptic  interactions 
with  soft  objects,  possible  pressure  distribution,  and  slip  detection  are  described.  In  addition, 
the  applicability  of  the  systems  and  results  from  this  study  to  dexterous  robot  finger  design, 
biomechanical  visualization,  and  automated  fingerprint  identification  systems  is  discussed. 
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Chapter  2 


Softness  Discrimination 

2.1  Abstract 

The  human  ability  to  actively  discriminate  softness  was  measured.  Pairs  of  silicon  rubber 
specimens  with  compliances  from  0.293mm/N  to  0.635mm/N  were  randomly  presented  to 
estimate  the  .Just  Noticeable  Differonce(.JND)  of  the  subjects  in  softness  discrimination.  Two 
experimental  paradigms  with  three  different  finger  conditions  were  used  to  evaluate  the  mech¬ 
anisms  that  were  used  during  discrimination.  Three  finger  conditions  (normal,  finger  covered 
with  a  finger  cot,  and  finger  covered  with  a  rigid  thimble)  were  chosen  to  affect  the  sources 
of  information  available  to  the  subjects  for  making  the  discrimination.  With  finger  cot,  the 
detailed  ridge  level  mechanics  of  contact  was  filtered  by  the  thin  (0.06mm)  latex  finger  cot. 
With  thimble  on,  the  subject  had  to  rely  mostly  on  the  kinesthetic  information.  Two  forced- 
choice  discrimination  paradigms  were  the  use  of  the  index  finger  from  the  dominant  hand 
sequentially  (1I-2AFC)  and  the  use  of  both  index  fingers  from  both  hands  simultaneously 
(S-2AFC).  In  the  S-2AFC  experiments,  the  two  specimens  were  available  during  the  whole 
process  of  discrimination;  thus  the  effect  of  memory  was  expected  to  be  less  than  that  in 
the  1I-2AFC  experiments.  With  the  1I-2AFC  paradigm,  the  mean  JND  values  for  the  three 
subjects  were  6%,  7%,  and  17%  under  normal  condition,  4%,  5%,  and  14%  under  finger  cot 
condition,  and  43%,  50%,  and  70%  under  thimble  conditions.  Analysis  of  variance  showed 
that  results  from  normal  and  finger  cot  conditions  were  not  significantly  different.  With 
S-2AFC  paradigm,  the  mean  JND  values  dropped  for  both  normal  and  finger  cot  conditions 
as  expected  (less  memory  effect)  and  the  difference  was  significant  at  1%  level;  the  JND 
values  were  3%  to  5%  for  two  subjects  and  were  8%  to  10%  for  the  third.  On  the  other 
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hand,  the  resulting  JND  under  the  thimble  condition  increased.  .4na.lysis  of  force  profiles 
used  during  both  1I-2.4FC  and  S-‘2AFC  paradigms  indicated  that  t3'pical  contact  duration 
was  0.5  sec  to  1.0  sec.  The  peak  force  used  ranged  from  about  ION  to  40N.  Comparing  the 
peak  forces  used  on  the  two  specimens  in  the  S-2AFC  experiments  suggested  three  strategies 
that  might  have  been  used  by  the  subjects:  (1)  to  achieve  the  same  amount  of  deformation 
of  the  tw'o  specimens  by  exerting  higher  forces  on  the  harder  specimen,  (2)  to  compare  the 
contact  information  obtained  by  applying  the  same  amount  of  pressure  on  both  specimens 
(thus  higher  peak  forces  on  the  softer  specimen),  and  (3)  to  apply  the  same  amount  of  peak 
forces  on  both  specimens  and  compare  the  deformations. 

2.2  Introduction 

Mumans  use  their  hands  to  explore  and  manipulate  objects.  In  both  exploration  and  manip¬ 
ulation.  the  contact  forces  as  well  as  the  finger  positions  must  be  controlled.  In  addition, 
properties  of  the  object  such  as  compliance,  shape,  and  texture  are  sensed  and  used  to  avoid 
dropping  or  crushing  the  objects. 

Softness  is  a  subjective  measure  of  the  compliance  of  an  object.  To  assess  the  .softness 
of  an  object,  the  object  is  squeezed  with  the  hand  or.  indented  with  the  finger.  During  such 
a  process,  two  sources  of  information  are  available:  Tactile  information  is  gathered  from 
the  sensors  embedded  in  the  skin  near  the  contact  region;  kinesthetic  information  is  gener¬ 
ated  by  the  sensory  receptors  around  joints,  tendons  and  muscles  (Srinivasan  and  LaMotte 
[42][40][41]). 

Researchers  have  conducted  various  experiments  in  measuring  human  sensory  and  motor 
performance.  Early  studies  on  softness  perception  existed  in  the  form  of  stimuli  ranking,  scal¬ 
ing,  and  determination  of  psychophj'sical  laws.  For  example,  studies  have  been  carried  out  on 
the  skills  of  test  bakers[27],  cheese  maker  and  grader  craftsmanship,  and  on  scaling,  matching 
and  cross-modal  comparisons  of  softness  perception [20].  Roland  and  Ladegaard-Pedersen[35] 
found  that  the  discriminabilitj'  of  spring  strength  was  unaffected  by  local  anesthesia  of  skin 
and  joints  when  the  springs  w'ere  held  between  thumb  and  index  finger.  However,  not  much  is 
know-n  about  the  human  resolution  and  the  discrimination  mechanisms  with  rubber-like  soft 
objects.  Just  Noticeable  Difference  (JND)  is  a  common  measure  of  human  sensory  resolution 
and  has  been  measured  for  a  variety  of  physical  quantities.  The  JND  values  for  length  (Dur- 
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lach  et  al  [13]),  force  (Pang  et  al  [34]),  compliance  and  work  have  been  measured  by  various 
researchers.  Tan  et  al  [45]  have  measured  the  JND  in  compliance  between  two  rigid  plates, 
controlled  by  linear  motors,  grasped  between  the  thumb  and  index  finger.  They  found  that 
the  .JNDs  ranged  from  about  8%  to  99%  depending  on  whether  or  not  the  subjects  had  cues 
such  as  terminal  grasp  force  or  total  work  done.  .Jones  and  Hunter  [26]  performed  experiments 
where  subjects  actively  moved  the  forearm  about  the  elbow  joint  to  sense  the  compliance  of 
a  preprogrammed  linear  motor  attached  to  one  of  the  wrists,  and  to  match  its  compliance 
by  modifying  that  of  an  identical  linear  motor  attached  to  the  other  wrist.  They  estimated 
the  compliance  .JND  to  be  about  23%i  under  those  conditions.  Srinivasan  and  Chen  [39] 
measured  the  human  ability  to  control  the  normal  forces  of  contact  with  a  rigid  ]>late  under 
a.  variety  of  experimental  conditions  and  found  that  performance  degraded  in  the  absence  of 
tactile  information  and  visual  feedback.  Srinivasan  and  LaMotte  [42]  investigated  the  ability 
of  humans  to  tactually  discriminate  the  softness  of  elastic  objects  with  deformable  (rubber 
specimens)  and  rigid  surfaces  (spring  cells).  They  performed  three  types  of  experiments:  act¬ 
ive  touch  with  normal  finger,  active  touch  with  local  cutaneous  anesthesia,  and  pa.ssive  touch, 
to  measure  the  human  discriminability  and  to  isolate  the  associated  information-processing 
mechanisms.  They  found  that  tactile  information  was  sufficient  for  discriminating  the  rubber 
specimens  used  in  their  study.  In  some  of  their  experiments,  the  peak  forces  were  constrained 
to  50,  75,  and  90  gwt  in  order  to  study  the  effect  of  force  on  discrimination. 

The  work  reported  here  was  designed  to  look  at  the  softness  discrimination  from  adifferent 
perspective.  The  subjects,  with  their  ey^es  closed,  were  allowed  to  use  any  force  to  discriminate 
the  specimens.  Two  experimental  paradigms  together  wdth  three  different  finger  conditions 
were  used  to  study  the  subjects’  JND  performance  on  softness  discrimination  for  rubber 
specimens. 

2.3  Methods 

2.3.1  Softness  specimen  preparation 

The  specimen  casting  procedure  was  similar  to  what  Srinivasan  and  LaMotte  [42]  used  in 
their  experiments,  but  the  mounting  method  used  here  w'as  improved  to  enable  high  volume 
production  of  the  specimens.  A  series  of  compliant  specimens  were  cast  by  mixing  different 
proportions  of  General  Electric  (GE)  SF96-50  silicon  rubber  diluent  to  fixed  proportions  of 
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GE  RTV615A  and  RTV615B  components.  The  diluent  GE  SF96-50  is  a  replacement  for  the 
discontinued  GE  RTV910  used  by  Srinivasan  and  LaMotte. 

The  component  proportions  were  determined  b}'  weight  using  a  digital  scale  and  dispensed 
via  pipet.  The  mixtures  were  thoroughly  mixed  by  hand  and  then  poured  into  30  mm  diameter 
petri  dishes  to  form  transparent  specimens.  Depending  on  the  proportions  of  the  mixture,  it 
took  a.s  long  as  24  hours  for  some  of  the  specimens  to  cure  completely.  At  least  5  specimens 
of  the  same  diluent  i)roportion  were  cast  to  allow  for  wear  and  tear  from  the  experiments  and 
for  force  control  and  contact  visualization  studies.  The  specimens  were  visually  identical  and 
were  mainly  distinguished  by  the  label  on  the  side  which  indicated  their  batch  and  sequence 
code. 

Several  batches  of  specimens  were  made  for  the  pilot  experiments  on  ranking  softness  in 
order  to  determine  how  close  the  compliance  of  the  specimens  should  be.  It  turned  out  that 
specimens  with  much  finer  differences  than  those  that  Srinivasan  and  LaMotte  used  had  to 
be  prepared  for  the  .JND  experiments.  The  standard  specimen  chosen  for  the  experiment  had 
an  objective  compliance  (.see  next  section)  of  0.30  mm/N. 

2.3.2  Specimen  objective  compliance 

An  objective  measure  of  the  specimen  compliance  was  obtained  by  the  characteristic  force- 
displacement  relationship  of  each  specimen.  A  linear  stepper  motor  system  with  a  micro¬ 
stepping  drive  was  used  to  indent  the  specimens  at  a  constant  velocity  of  0.5  mm/sec  with 
the  flat  end  of  a  1/4  inch  diameter  circular  probe.  As  the  probe  indented  the  specimen,  a. 
force  sensor  mounted  between  the  probe  and  the  motor  was  used  to  measure  the  indenta¬ 
tion  force  at  1000  samples  per  second  data  rate.  Since  the  stepper  motor  movement  was 
completely  determined  by  the  number  of  digital  pulses  received,  its  relative  position  can  be 
determined  to  better  than  1/rm  accuracy.  The  indentations  were  repeated  three  times,  and  the 
depth  of  indentation  was  3  mm  for  the  soft  specimens  and  smaller  for  the  harder  specimens. 
The  resulting  force-displacement  curves  were  then  used  to  compute  the  specimen  objective 
compliance  by  measuring  the  slope  of  a  fitted  straight  line  with  least  square  error. 

The  objective  compliance  value  computed  from  the  characteristic  line  using  least  square 
line  fitting  is  significant  to  the  third  decimal  place  or  0.3%  of  the  compliance  value  for  the 
specimen  with  the  lowest  objective  compliance.  The  resulting  values  from  the  three  repetitions 
were  averaged  and  used  as  the  specimen  compliance.  Figure  2-1  shows  the  characteristic 
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Compliance  Calibration - CR1  =  0.3041  mm/N 


Figure  2-1:  Objective  compliance  data  from  three  repetitions  of  the  prol)e  indenting  the 
standard  specimen  CRl.  The  numbers  shown  are  the  computed  objective  compliances  in  the 
three  trials. 

curves  for  the  standard  specimen  CRl  which  shows  that  the  force-displacement  data,  are 
very  repeatable  and  linear  to  a  high  precision.  The  specimens  used  in  all  the  discrimination 
experiments  had  compliances  that  ranged  from  0.293mm/N  to  0.635mm/N.  Table  2.1  lists 
the  objective  compliance  of  the  batches  of  specimens  used  in  the  discrimination  experiments. 
Serial  number  shown  indicates  specimens  with  identical  diluent  proportions. 

2.3.3  Specimen  presenter 

The  measurement  of  abilities  of  human  subjects  requires  experiments  with  a  large  number 
of  trials.  For  conducting  such  experiments  efficiently,  a  specimen  presenter  was  designed.  In 
essence,  the  specimen  presenter  is  a  stepper  motor  based  system  controlled  by  a  computer. 
It  is  capable  of  recording  the  forces  used  by  subjects  during  trials  and  switching  specimens 
between  trials.  The  computer  determined  randomly  how  specimens  were  presented  to  the 
subject  with  a  pre-determined  probability. 

A  block  diagram  of  the  specimen  presenter  system  is  shown  in  Figure  2-2.  The  specimens 
were  mounted  vertically  inside  two  specimen  holders.  The  holder  is  slightly  tapered  to  match 
the  shape  of  the  petri  dishes.  Its  bottom  has  a  circular  opening  but  does  not  allow  the 
specimen  to  fall  through.  The  specimens  fit  nicely  tight  and  can  also  be  taken  out  without 


23 


Batch  Code 

Serial  Number 

1  2  3  4  5  6 

CR 

0.304 

0.298 

0.300 

0.294 

DR 

Esa 

EJBl 

0.326 

0.324 

ER 

0.345 

0.348 

0.340 

FR 

0.382 

0.384 

■tlbwt* 

0.375 

0.380 

HT 

0.450 

0.416 

0.420 

0.442 

0.450 

IT 

0.523 

0.514 

0.518 

0..528 

.IT 

0.635 

0..598 

0.610 

0..587 

Table  2.1:  Specimen  Ol^jective  Compliance  (mm/N).  Serial  number  shown  refers  to  s])ecimens 
with  identical  diluent  proportions. 

too  much  effort.  * 

.4n  Eastern  .4ir  Devices  model  L.423BCK-01  stepper  motor  was  used  to  rotate  the  spe¬ 
cimens.  The  motion  of  the  motor  was  controlled  from  the  parallel  input/output  interface  of 
the  computer.  The  motor  control  signals  were  fed  to  a  model  7006-DB  motor  driver  from 
.4merican  Scientific  Instruments  Corp.  (AMSI)  which  drove  the  motor.  Two  custom  made 
strain  gage  based  force  sensors  were  designed  and  fabricated  to  measure  the  forces  exerted 
by  the  subjects  on  the  specimens.  A  finite  element  analysis  was  conducted,  by  using  .4DINA 
software,  to  compare  the  responses  to  a  point  load  and  a  distributed  load  from  three  differ¬ 
ent  force  sensor  designs:  simple  cantilever  beam,  cantilever  beam  with  a  hole,  and  cantilever 
beam  with  two  holes  and  a  slot  (binocular).  Binocular  spring  element  was  chosen  to  con¬ 
struct  the  sensors  because  it  is  more  rigid  and  less  sensitive  to  location  of  force  application. 
Four  strain  gages,  made  by  Measurements  Group,  were  mounted  on  the  top  and  bottom 
surfaces  of  each  force  sensor  near  the  center  of  the  two  holes  of  the  binocular  to  form  a  full 
bridge.  The  force  signals  were  amplified  by  a  signal  conditioner  which  consisted  of  an  analog 
module  rack  from  AMSI  and  two  Analog  Device  model  5B38-05  strain  gage  input  modules. 
These  amplified  force  sensor  signals  were  then  digitized  at  1000  samples  per  second  rate  by 
a  RTD-ADA3100  data  acquisition  board  made  by  Real  Time  Devices  Inc.  under  the  control 
of  custom  developed  C  programs. 

The  choice  of  having  the  specimens  mounted  vertically  on  the  motor  has  two  advantages. 


‘The  machining  of  the  force  sensors  and  the  specimen  holder  cis  well  as  other  attachments  for  the  experi¬ 
ments  was  carried  out  at  MIT  ME  student  shop  under  the  supervision  of  Norman  Berube,  The  author  wishes 
to  thank  him  for  his  patient  guidance  and  advice  on  machining. 
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Figure  2-2:  The  softness  specimen  presenter  system. 


First,  since  the  motion  of  the  finger  during  perception  will  be  horizontal,  the  gravity  effect 
will  be  the  same  for  the  chosen  experimental  paradigms.  Second,  the  force  sensor  offset  value 
need  only  be  measured  once  and  can  be  applied  to  the  two  possible  specimen  positions  used 
in  the  experiments. 

2.3.4  Finger  conditions 

To  further  our  understanding  of  the  underlying  mechanics  of  the  softness  discrimination  task, 
three  finger  conditions. were  used:  normal  finger,  finger  covered  with  a  0.06mm  thick  latex 
finger  cot,  and  finger  covered  with  a  rigid  thimble. 

With  normal  finger,  subjects  have  both  tactile  and  kinesthetic  information  and  possibly 
the  surface  texture  information  if  there  is  any.  When  the  finger  is  covered  with  a  thin  latex 
finger  cot,  one  can  still  feel  the  surface  easily.  It  is  similar  to  the  glove  worn  by  surgeons, 
except  that  the  finger  cot  here  was  just  for  a  single  finger.  The  compliance  characteristics 
of  the  finger  with  and  without  finger  cot  is  similar  as  shown  in  Figure  2-3.  However,  the 
finger  ridge  level  contact  information  is  degraded.  When  the  finger  wears  a  rigid  thimble,  the 
deformation  of  the  finger  and  the  deformation  of  the  specimen  depends  on  the  contact  force 
alone.  The  interactions  between  the  finger  and  the  specimen  is  replaced  by  the  interactions 
between  thimble  with  the  specimen  and  with  the  finger. 
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Normal  Finger  (solid),  Finger  Cot  (dashdot) 


Figure  2-3:  The  force  displacement  curves  for  finger  witli  and  without  finger  cot. 

By  comparing  the  results  from  the  three  different  conditions,  we  can  infer  how  softness 
discrimination  is  affected  l>y  the  presence  of  information  from  the  three  different  information 
sources:  detailed  finger  ridge  contact  information,  degraded  tactile  information,  and  the 
kinesthetic  information. 

2.3.5  Experimental  paradigms 

Since  humans  interact  with  objects  with  multiple  fingers  most  of  the  time,  it  is  interesting 
to  look  into  how  humans  discriminate  softness  using  either  a  single  finger  or  two  fingers. 
Therefore,  two  paradigms  were  used:  single  finger  from  the  dominant  hand  and  one  finger 
from  each  hand. 

In  the  single-finger  case,  the  experiment  consisted  of  a  one-interval,  two-alternative  forced 
choice  (1I-2AFC)  paradigm  with  correct  answer  feedback.  The  subject  moved  the  index 
finger  to  indent  the  specimen  mounted  on  the  specimen  presenter  while  the  rest  of  the  fingers 
grasped  a  fix  rod  for  achieving  a  standard  posture.  The  subject  had  to  decide  whether  the 
specimen  presented  was  the  softer  or  the  harder  one  of  the  two.  In  the  two-finger  case,  the 
experiment  consisted  of  a  two  finger  simultaneous,  two- alter  native  forced-choice  (S-2AFC) 
paradigm  with  correct  answer  feedback.  The  two  index  fingers  indented  the  two  specimens 
mounted  on  the  specimen  presenter  and  the  subject  had  to  decide  whether  the  left  or  the 
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Figure  2-4:  Relative  positions  of  the  fingers  and  the  specimens  for  the  two  experimental 
paradigms. 

right  one  was  the  softer  specimen  of  the  two.  Figure  2-4  shows  the  relative  position  of  the 
specimens  and  the  fingers  in  the  two  experimental  paradigms. 

One  of  the  significant  differences  between  the  two  paradigms  is  the  information  available 
to  the  subject.  In  the  1I-2.4FC  case,  subjects  must  remember  how  specimens  felt  like  in  order 
to  come  up  with  a  correct  response.  In  the  S-2AFC  case,  however,  both  specimens  were 
available  to  the  subject  during  the  discrimination.  Therefore,  the  subject  can  compare  both 
specimens  during  the  indentation.  This  will  be  an  easier  task  if  the  ability  to  control  and  sense 
at  each  hand  is  not  reduced  by  the  fact  that  both  hands  are  involved  at  the  same  time.  One 
might  expect  that  since  humans  use  multiple  fingers  on  a  daily  basis,  that  should  not  be  too 
bad  an  assumption.  Clearly,  we  can  squeeze  two  different  rubber  balls  and  distinguish  them. 
But,  can  we  squeeze  a  rubber  ball  composed  of  two  semi-spheres  with  different  material 
properties  using  fingers  from  two  different  hands  and  distinguish  their  relative  softness  ? 
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2.3.6  Experimental  procedures 


To  obtain  statistically  significant  results,  the  discrimination  experiments  require  a  substantial 
number  of  trials  for  each  subject.  One  standard  and  four  comparison  specimens  were  used 
for  each  experimental  condition.  The  comparisons  were  selected  depending  on  the  subject’s 
performance  and  the  finger  conditions.  Each  subject  was  tested  for  only  one  hour  per  day 
to  avoid  Iroredom  or  possible  injuries  from  repeated  motions.  Also,  finger  exercises  were 
provided  in  between  the  four  repeated  runs  of  64  trials.  The  first  run  was  used  a.s  the 
training  run  and  the  data  were  not  used  in  the  final  .JND  calculations.  However,  it  was 
conducted  just  like  a  normal  run. 

Before  each  run  started,  the  subject  was  given  an  opportunity  to  touch  the  two  specimens 
when  the>'  were  laying  on  the  bench.  Afterwards,  the  specimens  were  mounted  horizontally 
on  the  specimen  presenter.  The  subject  rested  their  elbow  on  a  fiat  surface  and  the  ]>alm 
grabbed  onto  a.  roughly  3/4  inch  diameter  wooden  rod  for  posture  reference  and  kept  their 
e.yes  closed.  .'\n  offset  value  of  the  force  sensor  was  taken  while  the  specimens  were  horizontal. 
.Afterwards,  the  motor  moved  to  its  vertical  home  position  with  the  hard  specimen  located  on 
the  top  and  the  soft  specimen  at  the  bottom.  Depending  on  the  random  number  generated, 
the  motor  rotated  either  clockwise  or  counterclockwise  with  an  equal  probability.  Thus  either 
the  soft  or  the  hard  specimen  was  located  at  the  right  after  the  motor  stopped.  During  the  64 
trials,  the  program  controlled  the  motor  to  go  to  the  two  possible  test  configurations  32  times 
each.  .After  the  motor  reached  its  destination  for  a  particular  trial,  a  delay  of  0.5  second  was  in 
effect  to  avoid  any  vibration  which  might  cause  incorrect  force  sensor  reading.  Afterwards, 
the  computer  speaker  beeped  to  prompt  the  subject  to  move  the  finger  for  indenting  the 
specimen. 

A  circular  buffer  was  used  in  the  force  data  collection  program  which  had  a  threshold  force 
of  0.5N.  A  total  of  two  seconds  of  force  data  was  collected  for  each  trial,  which  consisted 
of  0.5  seconds  before  0.5N  force  was  reached  and  1.5  seconds  after.  This  data  collection 
scheme  allowed  force  data  from  most  of  the  trials  to  be  recorded  appropriately.  Occasionally, 
however,  the  subject  would  make  multiple  contacts  using  forces  above  the  threshold;  the 
data  collection  was  triggered  on  the  first  significant  contact  which  might  not  have  all  the 
appropriate  data.  Or,  the  subject  might  start  indenting  the  specimen  after  the  motor  stopped 
but  a  little  before  the  computer  beeped.  In  that  case  (less  than  1%  of  the  trials),  the  force 


28 


trace  showed  a  sudden  jump  from  zero  at  time  equal  to  0.5  second. 


2.3.7  Data  analysis 

111  the  discrimination  experiments,  there  were  two  stimuli  5i,  S2  and  two  possible  responses 
/?i,  /?2-  If  fij  represents  the  number  of  trials  the  subject  responded  with  Ri  to  a  stimulus 
Sj,  the  results  can  be  arranged  in  the  following  2  by  2  confusion  matrix  format: 


Ri 

R2 

.5', 

fn 

1 12 

.S'2 

In 

The  sum  of  the  elements  //,  should  add  up  to  64  for  one  run.  If  the  subjecl  was  100% 
correct,  then  fn  =  f22  =  32  and  /12  =  /21  =  0.  If  the  subject  made  mistakes,  the  off- 
diagonal  elements  would  be  higher.  By  applying  the  decision  model  that  is  used  extensively 
in  psychophysics  [12]  based  on  signal  detection  theory  [44],  the  sensitivity  index  d'  and  bias 
/3  of  the  subject  can  be  quantified.  To  get  more  reliable  estimates,  the  data  from  the  three 
runs  (excluding  the  training  run)  were  pooled  together  for  such  calculations.  The  .JND  of 
softness  is  the  difference  in  the  stimuli  (percentage  difference  of  the  objective  compliance) 
that  corresponds  to  a  d'  value  of  1  and  is  estimated  by  dividing  the  percentage  difference  by 
the  calculated  sensitivity  index  d' .  To  combine  the  data  from  different  comparison  softness, 
the  JND  was  obtained  by  employing  the  standard  procedure  of  averaging  the  four  individual 
JND  estimates. 

The  analysis  of  variance  (ANOVA)  method  was  used  to  find  out  whether  the  results  were 
influenced  by  factors  such  as  finger  conditions  and  experimental  paradigms.  The  individual 
JND  estimates  from  each  comparison  specimen  run  was  used  for  the  calculations  of  the 
ANOVA  table.  To  compare  any  two  JND  results  obtained  under  different  circumstances, 
the  Student's  t  test  [31][24][36]  was  used.  The  statistical  analysis  was  performed  by  using 
MATLAB  statistics  toolbox.  The  results  from  different  subjects  were  not  averaged  in  order 
to  observe  their  differences  and  to  analyze  why  such  differences  existed. 
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Specimen 

DRl 

DR5 

ERl 

FRl 

Difference 

2.6% 

7.1% 

13.5% 

31.7% 

Confusion 

55  41 

74  22 

74  22 

92  4 

Matrix 

29  67 

20  76 

17  79 

1  95 

d' 

0.70 

1.55 

1.67 

4.04 

l3 

-0.17 

-0.04 

-0.09 

-0.29 

JND  (%) 

3.8 

4.6 

8.1 

7.8 

MeanJ  N  D  =  6.1% 

Table  2.2;  11-2AFC  discrimination  results  for  subject  CH  under  normal  condition  (Standard: 
CRl  [0.304mm/N]) 

2.4  Results 

2.4.1  1I-2AFC  discrimination 

In  the  1I-2AFC  discrimination,  the  subjects  used  the  index  finger  from  their  dominant  hand  to 
indent  the  specimen.  Three  subjects  participated  in  the  experiments:  CH(male),  CT(female), 
and  .JK  (female).  Three  finger  conditions  were  tested  to  see  how  different  contact  interface 
conditions  affect  the  performance  of  the  subjects.  The  batches  of  trials  for  different  conditions 
and  paradigms  were  randomized  to  minimize  training  effects. 

JND  results 

The  discrimination  results  from  an  experimental  run  was  represented  bj'  a  confusion  matrix. 
The  calculation  of  the  sensitivity  index,  d' ,  response  bias  /?,  and  JND  for  a  given  comparison 
specimen  was  performed  by  using  the  procedure  outlined  in  the  Methods  section. 

For  example,  table  2.2  shows  the  discrimination  data  and  the  calculated  JND  value  for 
subject  CH  under  normal  condition.  Specimen  CRl  was  used  as  the  standard  and  the  four 
comparison  specimens  and  the  differences  in  objective  compliance  from  the  standard  are 
listed.  The  confusion  matrix  was  pooled  from  the  three  runs.  The  sensitivity  index  as  well 
as  the  response  bias  was  calculated  according  to  the  decision  theory.  The  d'  value  obtained 
was  then  used  to  estimate  a  JND  value  by  dividing  the  percentage  difference  in  specimen 
compliance  by  d' .  Finally,  the  JND  under  normal  condition  for  subject  CH  was  obtained  bj' 
taking  the  mean  of  the  JND  estimates.  Figure  2-5  shows  how  the  sensitivity  index  d'  varies 
with  the  difference  in  objective  compliance  of  the  comparison  specimen.  A  least  square  line 
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Figure  2-5:  A  plol  of  .sensitivity  index  (d')  and  the  specimen  objective  compliance  difference 
for  subject  CH  under  normal  condition. 

was  drawn  to  show  that  the  relationship  was  approximately  linear. 

.4  summary  table  for  the  .JND  values  calculated  using  this  method  under  various  conditions 
for  all  subjects  are  listed  in  Table  2.3  (Data,  used  for  these  calculations  are  presented  in 
Appendix  B).  With  a  thimble  on  the  finger,  the  subjects’  performance  was  a  lot  worse  than 
the  normal  and  finger  cot  conditions.  For  subject  CH,  the  .JND  increased  from  6.1%  under 
normal  condition  to  43.3%  under  thimble  condition.  Keeping  in  mind  the  spread  of  the  JND 
estimates  in  Table  2.2,  we  can  observe  that  subjects  CH  and  JK  had  similar  performance 
under  normal  and  finger  cot  conditions  .  The  JND  values  for  subject  CT  were  higher  than 
the  other  two  subjects  in  all  three  conditions.  The  JND  increased  from  17.4%  for  normal  and 
14%  for  finger  cot  to  70.2%  with  thimble  on.  Clearly,  with  thimble  on,  none  of  the  subjects 
were  able  to  perform  anywhere  close  to  what  they  could  under  either  normal  or  finger  cot 
conditions. 

Analysis  of  variance  (ANOVA)  was  used  to  test  for  the  effects  of  subjects  and  finger  con¬ 
ditions  and  Student's  t  tests  were  used  to  compare  the  performances  from  different  subjects 
under  normal  and  finger  cot  conditions.  There  was  a  significant  difference  between  the  sub¬ 
jects  at  1%  significance  level  (the  significance  is  the  probability  of  observing  the  given  result 
by  chance  given  that  the  hypothesis  is  true) .  However,  the  differences  between  normal  and 
finger  cot  conditions  and  the  interaction  between  subject  and  condition  factors  were  small. 
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Condition 

Subject 

Mean 

value 

CH 

CT 

JK 

Normal 

6.1 

17.4 

7.0 

10.2 

Finger  cot 

4.3 

14.0 

5.0 

7.8 

Thimble 

43.3 

70.2 

49.8 

54.4 

Table  2.3;  1I-2AFC  softness  discrimination  mean  JND  (%) 


Significant  differences  of  the  mean  .JND,  at  5%  significance  level,  were  found  between  subjects 
(CII,CT)  under  normal  condition  and  between  subjects  (CH,CT)  and  (.JK,CT)  under  finger 
cot  condition.  The  conij^lete  analysis  is  presented  in  Appendix  C. 

Force  profiles 

.‘V  few  raw  data  graphs  are  shown  here  to  illustrate  how  the  applied  forces  varied  during  a  trial 
and  how  they  varied  across  trials,  conditions,  and  subjects.  Figure  2-6  shows  the  force  traces 
during  a  discrimination  run  that  consisted  of  64  trials.  In  that  particular  run,  the  specimens 
used  were  ERl  (comparison)  and  CRl  (standard)  and  the  finger  condition  was  normal.  Even 
though  the  subject  got  80%  correct  calls  in  the  discrimination,  the  force  traces  varied  from 
trial  to  trial  with  a  peak  force  ranging  from  ION  to  16N.  The  force  traces  intersect  at  0.5 
sec  time  mark  with  a  force  magnitude  of  0.5N  because  of  the  force  data  collection  mechanism 
e.xplained  in  the  experimental  procedure.  The  duration  of  contact  was  about  500  msec  to 
800  msec  during  that  run  and  the  indentation  phase  seems  to  take  a  little  longer  than  the 
releasing  phase. 

Figure  2-7  shows  the  same  force  traces  from  the  64  trials  separated  according  to  the 
stimuli  and  responses  (the  confusion  matrix  format).  The  title  for  each  panel  indicates  which 
stimulus  (S)  and  response  (R)  group  the  force  traces  came  from.  The  upper  left  and  the 
lower  right  graphs  show  the  forces  from  the  correct  responses  while  the  upper  right  and  the 
lower  left  graphs  show  the  forces  from  the  incorrect  responses.  The  numbers  on  the  upper 
left  corner  of  each  panel  indicates  the  number  of  force  traces  in  that  panel.  Those  numbers 
together  would  make  up  a  confusion  matrix.  The  top  row  shows  the  forces  used  for  the 
softer  specimen  and  the  bottom  row  shows  those  used  for  the  harder  specimen.  For  the  same 
specimen  (panels  in  the  same  row),  the  subject  used  forces  which  varied  from  trial  to  trial. 
The  force  traces  from  the  incorrect  responses  were  similar  to  the  ones  from  correct  responses. 
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Percentage  Correct:  0.80 
18  -  Data  File:  cher1cri;fn2 
Max  Peak:  16.2N 
Min  Peak:  10.3N 
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Figure  2-6:  Force  traces  from  subject  CH  during  a  1I-2AFC  discrimination  run  under  normal 
condition. 
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Figure  2-7:  Force  traces  from  subject  CH  during  a  11-2 AFC  discrimination  run  under  normal 
condition  plotted  in  the  confusion  matrix  format. 
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Figure  2-8:  Force  traces  from  subject  CH  during  a  1I-2AFC  discrimination  run  under  thimble 
condition. 


We  cannot  conclude  that  the  subject  responded  incorrectly  due  to  the  force  profiles  applied. 

The  presence  of  finger  cot  and  thimble  did  not  seem  to  have  much  effect  on  the  general 
shape  of  the  force  profiles  for  subject  CH.  Figure,  2-8  shows  the  force  profiles  from  an  exper¬ 
imental  run  with  the  thimble  condition  during  discrimination  of  the  easiest  specimen  pair  for 
that  condition.  Comparing  with  Figure  2-7  for  normal  finger,  the  general  shape  of  the  traces 
looked  about  the  same.  However,  the  forces  used  were  somewhat  higher  than  those  used  in 
the  normal  condition. 

Different  subjects  showed  quite  different  force  profiles  in  terms  of  peak  forces  used,  dur¬ 
ation  of  contact,  and  consistency.  Figure  2-9  and  Figure  2-10  show  typical  force  traces  from 
subject  CT  and  JK  under  normal,  finger  cot,  and  thimble  conditions.  For  subject  CT,  the 
shape  of  the  forces  was  not  as  consistent  as  those  of  subjects  CH  and  JK.  Also,  the  duration 
of  contact,  about  0.8  sec  to  1  sec,  was  longer  than  what  CH  and  JK  used  (about  0.5  to  0.8 
seconds).  The  force  profiles  exerted  on  the  specimens  by  subject  JK  was  quite  consistent 
and  the  peak  forces  during  each  trial  were  generally  higher  than  that  used  by  the  other  two 
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Figure  2-9:  Force  traces  from  subject  CT  during  1I-2AFC  discrimination  runs  under  nor¬ 
mal  (top  graph),  finger  cot(middle  graph),  and  thimble  (bottom  graph)  conditions. 
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Cond.:  Normal,  Data;  jker1cr1.fn3,  Specimen  Difference:  13.5%,  Correct;  84% 
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Cond.:  Finger  cot.  Data:  jker2cr2.fc3,  Specimen  Difference:  16.7%,  Correct:  86% 
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Cond.:  Thimble,  Data:  jkit2cr4.ft3.  Specimen  Difference:  75.2%,  Correct:  69% 
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Figure  2-10;  Force  traces  from  subject  JK  during  1I-2AFC  discrimination  runs  under  nor- 
mal(top  graph),  finger  cot(middle  graph),  and  thimble  (bottom  graph)  conditions. 
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subjects. 


2.4.2  S-2AFC  discrimination 

In  the  S-2AFC  discrimination,  the  subjects  simultaneously  used  the  index  fingers  from  both 
hands  to  perform  the  discrimination.  The  same  three  subjects  and  finger  conditions  were 
used.  Again,  the  purpose  of  using  three  finger  conditions  were  to  see  how  different  contact 
interface  conditions  affect  the  performance  of  the  subjects.  The  purpose  of  the  simultaneous 
contact  was  to  reduce  the  role  memory  plays,  since  both  specimens  were  available  during  the 
discrimination.  The  experiments  were  conducted  in  parallel  with  the  1I-2AFC  discrimination 
experiments  to  reduce  possible  training  effects.  Since  subjects  applied  forces  on  both  speci¬ 
mens,  the  analysis  on  the  force  profiles  could  possiblj'  reveal  the  underlying  strategies  used 
by  the  subjects  for  discrimination. 

JND  results 

The  .JND  values  for  the  S-2AFC  experiments  listed  in  Table  2.4  were  obtained  by  using  the 
same  pooled  data  method  used  for  calculating  the  JNDs  for  the  1I-2AFC  experiments.  The 
JND  values  obtained  were  lower  than  the  1I-2AFC  case  under  both  normal  and  finger  cot 
conditions  for  all  three  subjects.  However,  the  JND  value  for  the  thimble  condition  was  higher 
than  the  1I-2AFC  case  for  each  of  the  three  subjects. 

Analj^sis  of  variance  (ANOVA)  was  performed  to  determine  the  effects  of  subjects  and 
finger  conditions  and  Student's  t  tests  were  used  to  compare  the  performances  from  different 
subjects  under  normal  and  finger  cot  conditions.  There  was  significant  difference  between 
the  subjects  at  1%  significance  level  as  indicated  by  the  high  F  ratio  from  the  ANOVA 
table.  However,  the  differences  between  normal  and  finger  cot  conditions  and  the  interactions 
of  subjects  and  conditions  were  small.  T-test  at  5%  significance  level  revealed  significant 
differences  of  the  mean  JND  between  subjects  (CH,CT),(JK,CT)  under  both  normal  and 
finger  cot  conditions.  The  complete  analysis  is  presented  in  Appendix  C. 

Force  profiles 

The  shape  of  the  force  profiles  used  by  the  subjects  as  well  as  the  magnitude  of  the  peak  forces 
are  of  interest.  Figure  2-11  shows  the  force  traces  from  subject  CH  during  a  discrimination  run 
under  normal  condition.  The  upper  graph  shows  the  forces  exerted  on  the  standard  specimen 
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Condition 

Subject 

Mean 

value 

CH 

CT 

.JK 

Normal 

4.9 

HOI 

3.1 

Finger  cot 

2.9 

5.5 

Thimble 

57.9 

78.4 

56.1 

64.1 

Table  2.4:  S-2AFC  .softnes.s  discrimination  mean  JND  (%) 


(CR3)  whereas  the  lower  graph  shows  the  forces  applied  on  the  comparison  specimen.  Like 
the  1I-2AFC  experiments,  the  shapes  of  force  profiles  vaiy  from  trial  to  trial.  The  first  9 
trials  of  that  run  is  shown  separately  in  Figure  2-12  with  the  trial  number  labeled  in  the 
bottom  row'  of  upper  left  corner.  .4lso  shown  in  the  upper  left  corner  of  each  pane!  aie  the 
softer  stimulus  location  (labeled  S)  and  the  response  of  the  subject  (labeled  R).  The  solid  line 
represents  forces  on  the  comparison  specimen  (the  softer  one)  and  the  dashed  line  represents 
the  forces  on  the  standard.  In  the  first  trial,  the  softer  specimen  was  located  on  the  left  hand 
side  and  the  subject  correctly  responded.  The  moment  of  contact  on  the  standard  specimen 
was  earlier  than  for  the  comparison  (.softer)  specimen  for  trial  1.  Some  of  the  peak  forces 
w'ere  about  the  same  for  both  specimens  (trials  2  and  3)  while  some  were  higher  on  the  softer 
specimen  (solid  line,  trials  1,4, 7, 8, 9)  and  others  higher  on  the  harder  specimen  (dashed  line, 
trials  5,6).  The  moment  of  contact  was  sometimes  earlier  for  the  softer  specimen  and  other 
times  earlier  for  the  harder  specimen. 

To  see  whether  or  not  the  magnitudes  of  the  peak  forces  exerted  by  the  two  fingers  on 
specimens  w'ere  related  to  the  correctness  of  discrimination,  a  plot  of  the  peak  forces  on  the  soft 
specimen  versus  that  on  the  hard  specimen  for  subject  CH  under  normal  condition  w'as  shown 
in  Figure  2-13.  Symbols  ‘o’  and  ‘x’  were  used  for  correct  and  incorrect  responses,  respective!}'. 
The  four  panels  are  for  the  four  different  comparison  specimens.  The  differences  in  objective 
compliances  for  the  specimen  pairs  are  shown  on  top  of  each  graph.  The  percentage  correct 
(pooled  from  3  runs)  and  the  confusion  matrix  (labeled  CM)  of  the  subject  are  shown  in 
the  upper  left  corner.  The  numbers  labeled  FsM  are  the  elements  of  the  following  force- 
specimen-correctness  matrix  that  arranges  the  correct  and  incorrect  responses  according  to 
whether  or  not  the  peak  force  on  the  hard  (standard)  specimen  is  higher  than  that  on  the  soft 
(comparison)  specimen. 
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Time  (Sec) 

Figure  2-11:  Forces  on  hard  and  soft  specimens  from  subject  CH  during  a  S-2AFC  discrim 
illation  run  under  normal  condition. 


Figure  2-12:  Forces  exerted  on  soft  (solid  line)  and  hard  (dashed  line)  from  the  first  9  trials 
of  subject  CH  during  a  S-2AFC  discrimination  run  under  normal  condition. 
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Correct  Incorrect 
Phard  ^  Fsojt  ^11  -^12 

Fhard  ^  Psoft  ^21  ^22 

If  the  objective  of  the  subject  was  to  obtain  the  same  amount  of  deformation  from  both 
specimens  to  enable  a  discrimination  based  on  the  peak  force  applied  to  achieve  the  deform¬ 
ation,  then  the  subject  liad  to  exert  higher  forces  on  the  harder  specimen  pairs  and  higher 
force  on  the  hard  specimen  than  on  the  soft  specimen  for  a  given  pair.  Such  hypothesis  can 
be  tested  by  looking  at  the  FsM  matrix  and  comparing  the  occurrence  of  higher  forces  on 
the  hard  specimen  to  that  of  higher  forces  on  the  soft  specimen.  So,  we  can  sum  the  first 
two  numbers  of  the  Fs.M  matrix  which  were  the  number  of  trials  that  had  higher  jreak  forces 
on  the  hard  specimens  and  compare  it  to  the  sum  of  the  last  two  numbers  which  were  the 
number  of  t  rials  that  had  higher  peak  forces  on  the  soft  s])ecimens.  From  the  matrices  shown 
in  Figure  2- 13  for  subject  CH,  the  numbers  from  the  easiest  pair  to  the  most  difficult  pair 
were:  (97.95),  (94,98).  (106,86),  and  (111,81). 

For  subject  CT,  plot  shown  in  Figure  2-14,  the  opposite  was  observed.  The  numbers 
were  (75,117),  (81,111),  (84,108),  and  (98,94)  from  the  easiest  to  the  most  difficult  pairs. 
These  numbers  indicate  that  the  subject  used  higher  forces  for  the  softer  specimens  than  for 
the  harder  specimens  in  a  given  pair.  This  would  be  consistent  with  a  different  hypothesis, 
namely  that  the  subject  was  trying  to  obtain  the  same  amount  of  pressure  from  the  two 
specimens.  As  will  be  shown  in  contact  visualization,  for  a  given  force,  the  percentage  contact 
area  inside  the  contact  region  is  lower  for  harder  specimens.  Thus,  the  actual  pressures  for 
the  harder  specimens  are  higher  than  those  for  the  softer  specimens.  To  obtain  the  same 
amount  of  pressure,  therefore,  the  force  on  the  softer  specimen  should  be  higher  than  that  on 
the  harder  specimen.  As  the  difference  between  the  two  specimens  got  smaller,  the  applied 
peak  forces  also  got  closer  for  this  subject. 

For  subject  JK,  it  is  completely  different  from  the  other  two  subjects.  Unlike  the  plots 
from  subjects  CH  and  CT  which  had  data  points  clustered  around  the  45  degree  line  (the 
two  forces  did  not  differ  much),  the  data  points  separated  into  two  clusters  as  shown  in 
Figure  2-15.  When  the  difference  between  the  specimen  pair  was  large  (16.2%  difference), 
the  degree  of  separation  of  the  two  clusters  was  small.  For  the  most  difficult  pair  (3.0% 
difference),  the  degree  of  separation  was  largest.  Also,  subject  JK  used  peak  forces  that 
were  much  higher  than  the  other  pairs  and  saturated  the  input  of  the  data  acquisition  system 
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near  42N.  If  we  look  at  the  upper  cluster  of  the  3%  difference  specimen  pair,  most  of  the 
peak  forces  exerted  on  the  harder  specimen  were  within  the  range  of  22N  to  33N  while  most 
of  the  peak  forces  on  the  softer  specimen  were  more  than  40N.  The  numbers  from  the  FsM 
matrix  from  easier  to  difficult  specimen  pairs  were:  (95,97),  (96,96),  (96,96),  and  (96,96). 
It  was  very  consistent,  but  did  not  fit  either  of  the  two  hypothesis  so  far.  This  suggests  a 
third  hypothesis,  namely  that  the  subject  was  trying  to  exert  the  same  amount  of  force  and 
compare  the  contact  information  obtained  in  order  to  discriminate.  If  the  subject  was  indeed 
trying  to  exert  the  same  force  on  the  two  specimens,  the  fact  that  Figure  2-15  showed  data 
points  in  two  clusters  could  mean  the  subject  had  a  different  sensation  of  force  on  the  two 
fingers.  So,  if  we  plot  the  peak  forces  according  to  hands  instead  of  according  to  specimens, 
the  data  points  should  be  clustered  near  the  hand  which  used  higher  forces.  Figure  2-16 
shows  that  subject  .IK  used  higher  forces  from  the  right  hand  for  the  three  specimen  pairs 
that  are  more  difficult.  The  definition  of  the  FhM  matrix  is  similar  to  FsM  matrix  as  shown 
below: 
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For  this  subject,  summing  of  the  rows  of  the  FhM  matrix  in  a  manner  similar  to  that 
done  for  FsM  above  gives  (192,0),  (192,0),  (192,0),  and  (191,1).  These  results  clearly  show 
that  this  subject  almost  always  applied  higher  forces  with  the  right  hand. 

2,5  Discussion 

2.5.1  Effects  of  finger  conditions 

The  information  available  to  the  subjects  for  softness  discrimination  under  the  three  different 
finger  conditions  was  not  the  same.  With  normal  finger,  the  subject  had  detailed  tactile  as 
well  as  kinesthetic  information.  When  the  finger  were  covered  with  a  thin  rubber  finger  cot, 
the  subject  could  still  feel  the  surface  of  the  specimen  but  the  minute  finger  ridge  level  details 
were  degraded.  The  presence  of  finger  cot  acted  just  like  another  low-pass  filter  for  the 
mechanoreceptors  underneath  the  skin  which  originally  receive  information  already  filtered 
by  the  skin.  When  the  finger  had  a  rigid  thimble  on,  the  deformation  of  the  fingerpads  depend 
onl}'  on  the  interaction  force  with  thimble,  and  not  on  the  specimen  compliance.  Thus,  the 
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Figuie  2-13.  Peak  forces  on  soft  versus  hard  specimens  for  subject  CH  under  normal  condi 
tion . 
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Figure  2-14;  Peak  forces  on  soft  versus  hard  specimens  for  subject  CT  under  normal  condition 
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Figure  2-15:  Peak  forces  on  soft  versus  hard  specimens  for  subject  JK  under  normal  condition. 
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Figure  2-16;  Peak  forces  on  left  versus  right  specimens  for  subject  JK  under  normal  condition. 
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subjects’  sense  of  object  softness  will  have  to  come  mainly  from  the  kinesthetic  information. 
Because  all  the  experiments  involved  active  touch,  the  motor  commands  used  during  the 
discrimination  were  also  available  to  the  subjects  in  all  conditions. 

In  both  1I-2AFC  and  S-2AFC  experiments,  the  presence  of  the  thimble  degraded  the 
performance  of  all  three  subjects  dramatically.  The  general  shape  of  the  force  profiles  looked 
similar  to  the  ones  from  other  conditions.  However,  the  subjects  were  not  able  to  discriminate 
the  specimen  pairs  at  the  level  they  had  achieved  under  normal  or  finger  cot  conditions. 
Therefore,  even  though  the  subjects  were  able  to  control  their  forces  successfully  to  produce 
similar  force  profiles,  the  absence  of  useful  tactile  information  from  lack  of  direct  contact 
with  the  specimens  radically  degraded  their  ability  to  discriminate. 

Analysis  of  variance,  detailed  in  Appendix  C,  did  not  show  significant  difference  between 
the  .JND  results  obtained  under  normal  and  finger  cot  conditions.  This  implies  that  missing 
the  detailed  information  from  the  ridge  level  mechanics  of  contact  was  not  severe  enough  to 
degrade  the  softness  discrimination  performance  as  indicated  by  the  .JND  results. 

In  summary,  by  comparing  the  softness  discrimination  results  from  the  three  different 
conditions,  we  see  that  tactile  information  was  crucial  for  the  softness  discrimination  based 
on  the  poor  performance  with  the  thimble.  The  fact  that  the  presence  of  finger  cot  did  not 
significantly  affect  the  result  of  discrimination  suggests  that  the  information  from  the  fine 
finger  ridge  level  mechanics  of  contact  was  not  crucial  for  these  experiments. 

2.5.2  Effect  of  experimental  paradigms 

The  main  difference  between  the  two  experimental  paradigms  is  the  information  available 
to  the  subject.  In  the  S-2AFC  case,  both  specimens  were  available  to  the  subject  during 
the  discrimination.  In  the  1I-2AFC  case,  however,  subjects  had  to  remember  how  the  other 
specimen  felt  like  in  order  to  come  up  with  a  correct  response. 

The  JND  values  obtained  from  S-2AFC  paradigm  under  both  normal  and  finger  cot 
conditions  were  smaller  for  all  three  subjects.  This  is  consistent  with  our  expectation  since 
the  subjects  had  information  from  both  specimens  during  a  trial  and  thus  it  was  an  easier 
task.  The  ANOVA  analysis  in  Appendix  C  indicated  that  the  JND  data  obtained  from  the 
two  paradigms  are  significantly  different  at  1%  significance  level. 

From  the  box  plots  of  the  JND  data  from  the  two  paradigms  for  all  three  subjects  under 
both  normal  and  finger  cot  conditions  given  in  Appendix  C,  the  variations  of  the  data  from 
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the  S-2AFC  experiments  were  smaller  than  those  from  the  1I-2AFC  experiments.  We  cannot 
treat  the  results  from  the  one-interval  S-2AFC  experiments  just  like  a  traditional  2I-2AFC 
experiment.  The  improvement  in  performance  from  these  experiments  under  both  normal  and 
finger  cot  conditions  most  likely  should  be  attributed  to  the  elimination  of  the  role  memory 
played  in  the  discrimination. 

For  tlie  thimble  condition,  the  S-2.4FC  performance  was  worse  than  the  1I-2AFC  case. 
This  result  was  not  surprising  since  the  subjects  had  no  direct  contact  with  the  specimens  (no 
useful  tactile  information)  and  the  remaining  information  is  from  the  kinesthetic  sense  and  the 
knowledge  of  motor  commands  executed.  Any  tactile  information  received  could  potentially 
be  misleading  since  the  fingers  were  contacting  the  thimble,  not  the  specimens.  To  control 
two  fingers  carefully  and  at  the  same  time  to  ignore  the  misleading  tactile  information  would 
seem  like  a  tougher  job  for  subjects  and  it  showed  up  as  degraded  performance. 

2.5.3  Force  profiles 

The  force  profiles  used  by  subjects  during  the  discrimination  varied  across  subjects  and  trials. 
They  were  in  general  bell  shaped  curves  with  typical  contact  durations  between  0.5  sec  to  0.8 
sec  for  subjects  CH  and  .JK.  For  subject  CT.  the  contact  was  longer  and  ranging  from  0.8  sec 
to  1  sec.  The  profiles  did  not  seem  that  different  under  finger  cot  and  thimble  conditions.  The 
peak  forces  exerted  on  the  specimens  by  the  subjects  had  a  wide  range,  generally  between 
ION  and  40N.  Higher  forces  were  used  by  subject  JK  during  the  discrimination. 

The  force  profiles  exerted  on  the  two  specimens  looked  similar  in  the  S-2AFC  experiments 
with  two  fingers.  If  they  were  identical,  the  subject  might  be  able  to  discriminate  based  on 
the  resulting  deformation  of  the  fingers.  However,  if  the  force  profiles  from  the  two  fingers 
differed  such  that  the  resulting  deformation  of  the  two  fingers  were  the  same,  then  the  subject 
would  have  a  hard  time  discriminating.  For  example,  if  higher  force  rate  was  exerted  on  the 
harder  specimen  and  lower  force  rate  exerted  on  the  softer  specimen,  the  deformation  of  the 
hard  specimen  may  be  similar  or  more  than  that  of  the  softer  specimen.  In  that  case,  the 
subject  might  make  mistakes. 

A  longer  contact  duration  would  mean  lower  force  rates  and  if  that  information  was  used 
during  the  discrimination  then  it  would  explain  why  the  performance  of  subject  CT  was  not 
as  good  as  the  other  two  subjects. 
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2.5.4  Possible  softness  discrimination  mechanisms 


If  we  were  to  design  methodologies  for  a  robot  to  use  its  fingers  to  determine  the  compliance 
of  an  object,  we  can  have  strategies  that  utilize  force  and  displacement  information.  In  the 
force  based  strategy,  the  displacement  can  be  held  constant  by  pushing  against  an  object  to 
a  certain  deformation.  The  force  developed  during  the  contact  can  be  used  to  calculate  the 
compliance  of  the  object.  On  the  other  hand,  the  robot  can  push  against  the  object  until 
a  certain  maximum  force  is  reached.  In  that  case,  the  robot  finger  position  profile  together 
with  the  forces  at  every  instant  can  be  used  to  determine  the  object  compliance. 

In  humans,  investigating  the  underlining  mechanisms  is  more  complicated.  If  we  just  look 
at  the  force  profiles  from  the  iI-2AFC  discrimination  experiments,  we  would  be  lost  even  if 
we  break  down  the  trials  according  to  the  stimuli  presented  and  the  responses  given  by  the 
subject  (see  Figure  2-6).  .\lthough  the  shape  of  the  force  profiles  was  very  consistent,  the 
peak  forces  exerted  on  the  specimens  by  the  subjects  varied  from  trial  to  trial  and  had  a  wide 
range,  generally  between  ION  and  40N.  Also,  the  duration  of  contact  varied  from  trial  to  trial 
but  generally  ranged  from  0.5  sec  to  1.0  sec.  It  would  not  be  easy,  if  at  all  possible,  to  see 
the  underlying  softness  discrimination  strategies  used  by  the  subjects  from  the  1I-2AFC  force 
profiles.  However,  it  did  confirm  that  tactile  information,  degraded  (finger  cot  condition)  or 
not(normal  condition),  was  important  for  the  tasks  and  that  each  subject’s  ability  can  be 
significantly  different  from  the  others. 

By  looking  closely  into  the  peak  forces  in  the  force  profiles  used  by  the  subjects  during 
S-2AFC  experiments,  it  seems  quite  likely  that  the  subjects  were  using  different  strategies 
(see  S-2AFC  results).  For  subject  CH,  the  trend  was  to  exert  higher  forces  on  the  harder 
specimens.  This  would  be  consistent  with  trying  to  obtain  the  same  amount  of  contact 
information  (through  deforming  the  specimens)  from  both  fingers  and  thus  harder  specimen 
requires  higher  peak  forces.  For  subject  CT,  the  trend  was  to  use  higher  forces  on  the 
softer  specimen.  The  strategy  might  be  trying  to  apply  the  same  amount  of  pressure  on 
the  specimen.  For  softer  specimen,  higher  peak  forces  were  needed  because  the  percentage 
contact  area  inside  the  contact  region  was  higher  (see  Contact  Visualization  chapter).  Subject 
.IK  used  a  strategy  based  on  exerting  equal  amount  of  force  on  both  specimens.  However, 
subject  JK  exerted  very  different  forces  from  the  two  fingers  which  suggested  that  the  brain 
also  needs  to  compensate  for  such  differences  to  make  correct  discrimination. 
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The  lowest  JND  values  obtained  from  these  experiments  were  about  3%  by  subject  JK 
during  S-2AFC  discrimination  under  both  normal  and  finger  cot  conditions.  If  force,  dis¬ 
placement,  or  their  combination  were  the  only  cues  used,  such  small  JND  number  would  not 
be  possible  since  the  available  literature  report  higher  JND  values  for  those  physical  proper¬ 
ties  (see  Introduction).  However,  the  JND  values  reported  by  other  researchers  might  have 
inevitable  memory  effects  depending  on  the  paradigms  used.  On  the  other  hand,  the  subject 
could  also  be  using  other  types  of  information  to  achieve  the  discrimination.  Even  though  the 
forces  and  strategies  used  were  very  different  for  ail  subjects,  the  percentage  contact  area  in¬ 
side  the  contact  region  were  higher  for  softer  specimens  (see  Contact  Visualization  chapter). 
The  subjects  could  be  using  the  spatio-temporal  pressure  distribution  to  di.scriminatc  tlie 
specimens.  In  that  case,  the  finger  ridge  level  details  under  the  finger  cot  condition  might  not 
be  very  important  and  the  absence  of  useful  tactile  information  in  thimble  condition  would 
have  a  big  impact  on  the  JND  results. 

2.5.5  Important  factors  in  discrimination 

In  a  general  discrimination  task,  our  ability  to  discriminate  relies  on  sensing,  control,  and 
cognition.  It  is  only  by  conducting  experiments  under  carefully  designed  conditions  that  the 
relative  role  played  by  each  component  could  be  understood. 

In  the  softness  discrimination  tasks  reported  here,  the  finger  condition  was  a  key  factor 
that  had  significant  effect  on  the  performance.  Without  the  presence  of  useful  tactile  inform¬ 
ation,  the  task  was  difficult  as  shown  by  the  poor  thimble  performance.  The  finger  ridge 
levei  details  of  mechanics  of  contact,  however,  did  not  seem  to  be  important  in  the  softness 
discrimination  tasks  examined  here  as  shown  by  the  excellent  performance  of  subjects  under 
finger  cot  condition.  A  reasonable  explanation  lies  in  the  fact  that  the  mechanoreceptors 
are  underneath  the  skin  and  receive  only  low-pass  filtered  signals.  The  presence  of  the  thin 
(O.OGnim)  finger  cot  did  not  seem  to  change  the  input  to  the  receptors  much. 

By  conducting  S-2AFC  experiments,  we  had  an  opportunity  to  examine  the  underlying 
strategies  used  by  subjects  and  also  effectively  reduced  the  role  memory  played  in  the  dis¬ 
crimination.  The  peak  force  data  suggested  three  different  strategies  used  by  three  different 
subjects:  to  obtain  the  same  amount  of  deformation,  to  apply  the  same  amount  of  pressure, 
and  to  apply  the  same  amount  of  force.  No  matter  which  strategies  were  used  by  the  subjects, 
the  coordination  between  sensing,  control,  and  cognition  seems  to  be  important  for  successful 
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discrimination. 


Chapter  3 


Force  Control 

3.1  Abstract 

To  study  the  ability  of  humans  in  controlling  the  forces  of  contact  applied  by  the  finger- 
pads,  constant  normal  force  control  experiments  were  conducted  under  several  experimental 
conditions.  The  experimental  parameters  were  finger  contact  interface  conditions,  specimen 
softness,  target  normal  force  magnitude,  availability  of  visual  feedback,  and  the  fingers  from 
the  left  or  right  hand  of  the  subjects.  Three  finger  contact  interface  conditions  were  tested: 
normal  finger,  finger  covered  with  a  finger  cot,  and  finger  covered  with  a  rigid  thimble.  Four 
different  specimens  were  used:  a  rigid  Plexiglas  and  three  transparent  silicon  rubber  speci¬ 
mens  with  different  compliance  values.  Target  normal  force  magnitudes  of  2N,  4N,  and  8N 
were  used.  The  experiments  consisted  of  controlling  forces  during  three  stages,  5  seconds 
each,  of  force  feedback:  tracking  with  numerical  force  display,  tracking  with  graphical  force 
display,  and  maintaining  target  force  without  visual  feedback.  The  numerical  feedback  stage 
was  used  to  help  the  subjects  to  exert  forces  close  to  the  target,  and  therefore  the  perform¬ 
ance  was  not  analyzed;  only  the  data  from  graphical  feedback  stage  and  the  maintaining 
stage  were  analyzed.  Index  fingers  from  both  the  left  hand  and  right  hands  were  tested.  The 
statistical  tests  on  the  data  collected  indicate  that  the  finger  contact  condition  as  well  as  the 
specimen  softness  did  not  have  significant  effects  on  the  results.  Not  surprisingly,  when  there 
was  visual  force  feedback,  the  error  was  smaller  than  without  feedback  for  all  the  subjects. 
For  two  subjects,  the  percentage  error  increased  from  less  than  2%  at  8N  target  with  visual 
force  feedback  to  about  4%  and  6%  without  visual  force  feedback,  respectively.  For  the  other 
subject,  the  increase,  from  2%  to  11%  at  8N,  was  more  dramatic.  The  subject  who  had  the 
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highest  error  without  visual  feedback  was  the  same  one  who  had  higher  JND  values  (less 
able  to  discriminate)  in  the  softness  discrimination  experiments.  In  both  stages  analyzed,  the 
tests  on  the  handedness  showed  that  there  was  a  significant  difference  between  the  results 
obtained  for  the  two  hands  of  one  of  the  subjects  who  also  consistently  showed  handedness 
during  softness  discrimination  experiments. 

3.2  Introduction 

The  interaction  of  human  hands  with  objects  involves  both  sensing  and  control  of  the  contact 
interface.  The  human  abilities  of  tactual  perception  and  manipulation  are  de])endent  on  the 
haptic  system,  consisting  of  the  mechanical,  sensory,  motor,  and  cognitive  subsystems.  Tac¬ 
tual  sensory  information  concerning  contact  with  an  object  can  be  divided  into  tlirec  cla.ssos; 
(1)  tactile  information,  referring  to  the  sense  of  the  type  of  contact  with  the  object  as  well  as 
some  of  the  jrliysical  properties  of  the  object,  mediated  by  the  responses  of  receptors  innerv¬ 
ating  the  skin  within  and  around  the  contact  region;  (2)  kinesthetic  information,  referring  to 
the  sense  of  position  and  motion  of  body  segments  along  with  the  associated  forces,  conveyed 
by  the  sensory  receptors  in  the  skin  around  the  joints,  joint  capsules,  tendons,  and  muscles; 
(3)  neural  signals  derived  from  motor  commands.  The  motor  subsystem  complements  the 
sensory  system  by  enabling  control  of  body  postures  and  motions  together  with  the  forces  of 
contact  with  objects. 

In  performing  manual  tasks  in  real  or  virtual  environments,  contact  force  is  perhaps  the 
most  important  variable  that  affects  both  tactual  sensory  information  and  motor  performance. 
When  the  hand  is  actively  pressed  against  an  object,  the  contact  forces  are  sensed  by  both 
the  tactile  and  kinesthetic  sensory  systems.  The  Just  Noticeable  Difference  (JND)  in  contact 
force  is  about  7%  over  a  wide  range  of  conditions  involving  variations  in  force  magnitude, 
muscle  s,ystem  and  experimental  method,  provided  that  the  kinesthetic  sense  is  involved 
in  the  discrimination  task  (Pang  et  al[33]).  In  closely  related  experiments  consisting  of 
distinguishing  among  different  weights  of  objects,  a  slightly  higher  JND  of  about  10%  has 
been  observed  (see  reviews  by  Clark  and  Horch  [8]  and  Jones  [25]).  In  experiments  involving 
grasping  and  lifting  of  objects  using  a  two-finger  pinch  grasp,  Johansson  and  Westling  [22] 
have  shown  that  subjects  have  exquisite  control  over  maintaining  the  proper  ratio  between 
grasping  and  lifting  forces  (i.e.,  the  orientation  of  the  contact  force  vector),  so  that  the  objects 
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do  not  slip.  However,  when  tactile  information  was  blocked  using  local  anesthesia,  this  ability 
deteriorated  significantly  because  the  subjects  could  not  sense  contact  conditions  such  as  the 
occurrence  of  slip,  and  hence  did  not  apply  appropriate  compensating  grasp  forces.  Thus, 
performance  in  tasks  involving  contact  require  sensing  of  appropriate  forces  as  well  as  using 
them  to  control  contact  conditions.  Srinivasan  and  Chen  [39]  measured  the  human  abilities  to 
control  the  normal  forces  of  contact  exerted  by  the  finger[)ad  on  rigid  objects  under  a  variety 
of  experimental  conditions  and  found  that  performance  degraded  significantly  in  the  absence 
of  tactile  information  (by  using  local  anesthetized  fingertips)  and  visual  feedback  (by  using 
an  eye  mask). 

'Fhe  goal  in  this  study  was  to  measure  the  human  abilities  in  controlling  normal  force  of 
contacl.  applied  by  the  fingerpad  on  objects  of  various  softnesses  under  several  experimental 
condit  ions.  Three  contact  interface  conditions  were  used  to  study  the  effect  of  altered  tactile 
iid'ormation  on  force  control  performance.  The  results  of  control  with  visual  feedback  was 
compared  with  those  without  visually  feeding  back  the  applied  forces  to  the  subjects.  The 
same  experiments  were  performed  on  index  fingers  from  both  hands  of  three  subjects.  The 
absolute  error  between  the  target  force  and  the  forces  exerted  by  the  subjects  were  averaged 
over  three  trials  as  a  measure  of  human  performance.  In  addition,  the  effect  of  performances 
under  various  conditions  were  compared  using  anal.ysis  of  variance  (.A.NOVA). 

3.3  Methods 

3.3.1  Experimental  setup 

The  same  setup  used  for  the  softness  discrimination  experiments  was  used  for  the  force  control 
experiments  in  order  to  study  the  abilities  of  humans  in  controlling  normal  forces  of  contact 
with  specimens  of  different  softness.  An  illustrative  diagram  of  the  experimental  setup  is 
shown  in  Figure  3-1. 

The  soft  specimens(see  Softness  Discrimination  Methods)  were  mounted  vertically  inside 
one  of  the  specimen  holders  depending  on  the  finger  being  tested.  The  rigid  specimen  was 
mounted  on  top  of  the  specimen  holder.  The  soft  specimens  fitted  nicely  tight  in  the  specimen 
holder  and  could  also  be  taken  out  without  too  much  effort.  To  record  the  forces  exerted  on 
the  specimens,  a  pair  of  custom  made  binocular  force  sensors,  sufficiently  stiff  (see  Softness 
Discrimination  Methods)  and  not  sensitive  to  the  location  of  load  application  were  interfaced 
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Figure  3-1:  The  force  control  experimental  setup. 


to  an  analog  to  digital  (A/D)  conversion  board  in  a  80486  personal  computer.  The  forces 
exerted  on  the  specimens  were  displayed  differently  in  each  of  the  three  5-second  experimental 
stages  during  each  run.  The  force  data  as  well  as  the  force  display  were  updated  at  a  rate 
of  200  samples  per  second.  The  stepper  motor  was  controlled  through  digital  input/output 
(I/O)  port  and  was  energized  to  hold  the  specimen  at  a  vertical  configuration  during  the 
experiments.  The  subjects  were  sitting  comfortably  in  front  of  the  computer  monitor  with 
their  elbow  rested  and  hands  grasping  the  fixed  rods  provided  for  controlling  hand  posture 
during  the  experiments,  as  shown  in  Figure  3-2  for  left  and  right  hands. 


3.3.2  Experimental  parameters 

The  five  experimental  parameter's  used  were  contact  interface  conditions,  specimen  softness, 
target  normal  force  magnitude,  visual  feedback  availability,  and  the  fingers  used  for  experi¬ 
ments.  The  same  three  subjects  who  participated  in  the  softness  discrimination  experiments 
were  used  for  tests  on  all  parameter  combinations. 

In  order  to  determine  the  relative  importance  of  tactile  feedback  in  controlling  the  contact 
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force  on  the  fingerpad,  the  tracking  experiments  were  conducted  with  the  subject’s  finger 
under  normal  condition  (both  tactile  and  kinesthetic  information  available  to  the  subject), 
finger  cot  condition  (fine  finger  ridge  contact  information  is  degraded  by  the  thin  rubber 
finger  cot),  and  thimble  condition  (the  tactile  information  is  derived  from  fingerpad  contact 
with  a  rigid  thimble  instead  of  direct  fingerpad  contact  with  the  specimens). 

Four  different  specimens  were  used:  a  rigid  Plexiglas  plate  and  three  transparent  silicon 
rubber  specimens  with  different  softnesses.  The  soft  specimens  were  cast  by  mixing  different 
proportions  of  General  Electric  (GE)  SF96-50  silicon  rubber  diluent  to  fixed  proportions  of 
GE  RTV615A  and  RTV615B  components.  An  objective  measure  of  the  specimen  .softness 
was  measured  by  using  a  linear  stepper  motor  system  with  micro-stepping  drive  to  indent  the 
specimen  at  a  constant  velocity  of  0.5  mm/sec  with  the  flat  circular  end  of  a  1/4  inch  diameter 
cylindrical  probe.  The  characteristic  force-displacement  relationship  of  the  specimens  was 
linear(Figure  2-1).  The  throe  specimens  chosen  had  objective  compliance  of  0.304mm/N 
(hard),  0.455mm/N  (medium),  and  0.608mm/N  (soft).  The  variability  in  these  compliance 
measurement  was  only  ±0.001mm/N. 

Target  normal  force  magnitudes  of  2N,  4N,  and  8N  were  used  as  one  of  the  experimental 
parameters.  Although  8N  was  not  as  high  as  the  peak  forces  generally  applied  by  the  subjects 
during  tasks  such  as  softness  discrimination,  it  was  close  to  the  maximum  constant  force  that 
subjects  could  apply  and  still  be  comfortable  after  repeated  experiments  of  15  seconds  each 
for  an  hour.  The  other  two  reference  force  magnitudes  were  used  in  order  to  determine  how 
the  performance  varied  with  target  force  magnitude. 

The  experiments  consisted  of  applying  and  controlling  forces  with  ,  the  fingerpad  for  15 
seconds  under  three  stages  of  force  feedback:  tracking  with  a  numerical  force  display,  tracking 
with  graphical  force  display,  and  maintaining  target  force  without  visual  force  feedback. 
Subjects  began  the  force  application  in  each  trial  without  any  external  triggering  signal. 
During  the.  first  5  seconds,  the  force  used  by  the  subjects  was  displayed  on  the  screen  as  a 
number(unit;N).  During  the  next  5  seconds,  the  display  changed  to  a  graphical  display.  A 
horizontal  bar  extended  out  from  the  right(left)  edge  of  the  screen  was  shown  in  the  middle 
of  the  monitor  and  ended  at  a  distance  from  the  center  which  was  proportional  to  the  force 
error  for  trials  using  right(left)  hand.  The  subjects  were  instructed  to  try  and  maintain  the 
end  of  the  force  display  bar  to  be  as  close  as  possible  to  the  fixed  target  line  drawn  vertically 
in  the  middle  of  the  monitor.  During  the  last  5  seconds,  the  graphical  display  of  the  contact 
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force  was  deactivated  and  the  bar  displayed  did  not  change  according  to  the  force  applied. 
Subjects  had  to  maintain  the  target  force  based  on  their  manual  feel  of  it.  The  display  of  the 
forces  during  these  three  stages  are  illustrated  in  Figure  3-3  for  both  fingers. 

In  the  real  world,  we  manipulate  objects  with  fingers  from  both  hands.  To  investigate 
whether  or  not  there  is  a  significant  difference  in  the  control  of  forces  for  fingers  from  the 
two  hands,  the  same  tests  were  conducted  for  the  index  fingers  from  both  the  dominant  and 
non-dominant  hands. 

3.3.3  Experimental  procedures 

Since  there  are  too  many  parameters  for  the  experiment  to  be  randomized  between  paramet¬ 
ers.  the  experiments  were  conducted  with  a  fixed  parameter  sequence.  The  experiments  on 
the  two  different  fingers  were  conducted  in  two  different  sessions.  The  parameter  of  force 
feedback  method  was  varied  during  the  stages  of  each  experimental  run.  The  other  three 
parameters  were  varied  in  the  following  waj^  For  each  specimen  tested,  the  finger  contact 
interface  conditions  were  varied  first.  Under  each  finger  condition,  the  target  force  was  var¬ 
ied  from  2N  to  4N  and  finally  8N.  The  same  three  subjects  tested  for  softness  discrimination 
repeated  the  force  control  trials  at  each  target  force  level  three  times. 

Before  a  session  started,  the  subjects  were  given  practice  runs  with  both  hard  and  soft 
specimens  until  the  procedure  was  familiar.  Each  subject  was  seated  so  as  to  comfortably 
press  his/her  index  finger  on  specimens  attached  to  the  force  sensor.  The  posture  rod  in 
front  of  the  specimen  provided  a.  good  reference  position  for  the  subject’s  hand  and  finger 
during  the  experiments.  At  the  beginning  of  each  trial  a  target  force  was  shown  as  a  number 
in  Newtons  on  the  monitor  and  the  subject  was  also  informed  by  the  experimenter  about 
the  target  force  magnitude.  The  subject  started  the  indentation  on  his/her  own  without  any 
external  triggering  signals.  The  control  program  written  in  C  language  had  a  circular  buffer 
that  waited  for  the  moment  of  contact  to  happen  before  it  started  logging  the  force  data. 
After  the  subject  exerted  significant  amount  of  force  (0.5N)  on  the  specimen,  the  trial  began. 
During  the  first  stage,  the  applied  force  was  displayed  in  numerical  format  near  the  left  or 
right  side  of  the  screen  depending  on  whether  the  finger  from  the  left  or  right  hand  was 
involved  in  the  experiment.  The  first  stage,  which  lasted  5  seconds,  was  needed  to  help  the 
subject  get  to  the  target  force  level.  During  the  second  5  seconds,  the  force  was  fed  back  to 
the  subject  through  graphical  force  display  as  described  before.  A  bar  started  from  the  sides 
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Figure  3-3:  Contact  force  display  during  the  three  stages  of  force  control  experiments  for  left 
and  right  hands. 


of  the  screen  and  approached  the  center  of  the  screen  as  the  subjects  exerted  forces  closer 
to  the  target  magnitude.  After  some  pilot  experiments,  the  range  of  the  entire  display  was 
chosen  to  be  half  of  the  target  force  range  during  the  graphical  feedback  stage.  The  screen 
resolution  was  set  to  VGA  mode  with  640  Iw  480  pixels.  For  the  low  force  target  (2N),  each 
pixel  corresponds  to  about  0.0016N.  At  high  target  force  (8N),  the  force  display  resolution 
was  about  0.0063N  per  pixel.  Such  high  force  display  resolution  should  allow  the  subjects 
to  see  the  force  difference  clearly.  During  the  final  stage,  the  graphical  bar  occupied  half  the 
screen  and  remained  the  same  irrespective  of  the  forces  exerted  by  the  subjects.  In  this  stage, 
the  subject  was  instructed  to  maintain  the  target  force  for  5  seconds  based  on  memory.  After 
t  he  period  ended,  the  screen  changed  back  to  the  initial  state  for  the  next  run. 

3.3.4  Data  analysis 

Several  measures  of  the  force  error  between  the  target  force  and  the  forces  exerted  by  the 
subjects  were  computed:  (1)  the  standard  deviation  from  the  mean  force  exerted;  (2)  the 
absolute  error  with  respect  to  the  target  force;  (3)  the  absolute  error  expressed  as  a  percentage 
of  the  target  force.  The  results  described  below  are  given  in  terms  of  the  absolute  errors.  The 
force  data  sampled  at  200  samples  per  second  were  filtered  with  a  low-pass  filter  that  had  a 
cut-off  frequency  of  .50Hz  for  noise  removal  before  the  force  error  were  calculated.  The  mean 
absolute  error  and  the  pooled  standard  deviation  [24]  were  tabulated.  Also,  only  data  from 
the  second  and  third  stage  of  each  run  were  analyzed.  The  analysis  of  variance  (ANOVA) 
[31][24][36]  was  used  to  find  out  whether  the  results  were  significantly  influenced  by  various 
factors.  The  data  averaged  from  three  individual  trials  under  each  experimental  conditions 
were  used  for  the  calculations  of  the  ANOVA  table.  The  statistical  analysis  was  performed 
by  using  MATLAB  statistics  toolbox.  The  results  from  different  subjects  were  not  averaged 
in  order  to  observe  their  differences. 

3.4  Results 

3.4.1  Force  profiles 

Typical  force  exerted  by  the  subjects  during  the  experiments  are  shown  in  Figures  3-4,  3-5, 
and  3-6.  The  rows  of  panels  are  force  traces  on  different  specimens  whereas  the  different 
columns  are  for  different  finger  conditions.  Three  force  traces  are  shown  in  each  panel  for 
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the  three  target  forces  used.  Any  force  variations  at  frequencies  greater  than  about  50  Hz 
are  removed  by  low-pass  filtering  the  recorded  force  data. 

Visual  inspection  of  the  data  suggests  that  the  differences  between  data  obtained  from 
diiferent  specimens  are  similar.  The  tracking  force  from  the  second  stage  (5  to  10  seconds 
from  the  start)  of  each  run  was  quite  close  to  the  target,  especially  for  the  two  low  force 
targets.  However,  the  last  stage  (10  to  15  seconds  from  the  start)  of  the  force  traces  seemed 
to  have  more  variability  due  to  the  fact  that  no  visual  force  feedback  was  given.  For  exami^le, 
in  Figure  3-4,  the  forces  exerted  by  subject  CH  on  the  rigid  specimen  dropped  considerably 
during  the  third  stage  under  finger  cot  condition.  For  subject  CT,  the  visual  inspection 
on  some  trials  shown  in  Figure  3-5  suggested  that  the  data,  from  the  third  stage  (without 
visual  feedback)  wore  considerably  different  from  the  second  stage  (with  graphical  visual 
feedback).  In  almost  all  the  8N  target  trials,  the  force  traces  during  the  third  stage  dropped 
by  a  noticeable  amount,  sometimes  more  than  2N  (25%).  This  visual  inspection  suggested 
that  the  subject  CT  had  difficulty  maintaining  constant  force  without  visual  feedback.  For 
subject  .]K,  force  traces  from  the  third  stage  also  shows  considerable  deviation  from  the  target 
force,  but  were  not  as  severe  as  those  observed  for  subject  CT. 

Although  the  error  distributions  obtained  from  the  force  traces  would  depend  on  particular 
trials,  there  was  a  distinct  difference  between  the  distributions  obtained  from  the  second  and 
third  stage  for  most  trials.  Shown  in  Figure  3-7  are  distributions  of  error  and  absolute  error 
for  stage  2  and  stage  3  of  a  typical  run.  The  error  distribution  for  the  second  stage  was 
typically  shaped  close  to  a  normal  distribution  centered  around  zero.  However,  the  error- 
distribution  for  the  third  stage  was  usually  not  centered  around  zero.  The  corresponding 
absolute  error  distributions  were  shown  in  the  bottom  two  panels.  For  trials  with  continuing 
drift  in  the  third  stage,  the  error  distribution  would  be  quite  flat  and  distributed  on  one  side 
of  zero  (see  Figure  3-8). 

3.4.2  Performance  measurement 

Due  to  the  large  number  of  experimental  parameters  involved,  a  single  table  which  contains  all 
the  performance  data  is  not  possible.  Instead,  the  results  have  been  grouped  by  the  following 
three  factors:  subject,  visual  feedback  availability,  and  the  hands  used.  The  performance 
measurement  using  absolute  errors  for  each  subject  under  various  experimental  parameters 
are  given  in  Appendix  E. 
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Figure  3-4:  Typical  force  versus  time  traces  for  subject  CH  under  different  finger  conditions 
on  various  specimens 
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Figure  3-5:  Typical  force  versus  time  traces  for  subject  CT  under  different  finger  conditions 
on  various  specimens 
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Figure  3-7:  A  typical  error  and  absolute  error  distribution  for  the  two  stages  of  an  exper¬ 
imental  run  from  a  trial  conducted  on  rigid  specimen  with  2N  target  force  under  normal 
condition  by  subject  CH. 
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Figure  3-8:  A  typical  error  and  absolute  error  distribution  for  the  two  stages  of  an  exper¬ 
imental  run  from  a  trial  conducted  on  rigid  specimen  with  8N  target  force  under  normal 
condition  by  subject  CH. 
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Condition 

Force 

-Absolute  Error  (N) 

Hard 

Soft 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N 

0.035 

hiitifaa 

Bifigra 

0.030 

0.028 

0.036 

0.031 

Normal 

4N 

imbii 

IQBI 

■miwti 

0.056 

0.043 

Normal 

bb 

■IlIMM 

0.077 

0.061 

liligg 

lliltllti 

I'lIiMl 

■IWIKCT 

liTigM 

Finger  cot 

0.0.54 

liligol 

0.049 

0.037 

0.037 

Finger  cot 

8N 

■wisa 

0.108 

0.093 

■nresi 

Thimble 

2N 

itlihlfJ 

■iWtEftl 

Thimble 

IIQJIII 

0.048 

0.060 

iilihVJ 

gnngiii 

mmm 

0.087 

0.098 

0.0.59 

0.066 

0.0.53 

0.110 

0.077 

Table  3.1:  Absolute  error  in  force  control  for  subject  CH  using  right  hand  index  finger  with 
visual  feedback. 


The  tables  for  the  three  subjects  when  visual  feedback  was  present  and  the  right  hand 
was  used  are  shown  here  in  Tables  3.1,  3.2,  and  3.3.  The  mean  absolute  error  was  obtained 
by  averaging  the  mean  absolute  error  of  the  three  experimental  runs,  with  the  same  number 
of  samples,  under  the  same  conditions.  The  standard  deviation  (s)  was  obtained  by  pooling 
the  standard  deviations  from  the  three  experimental  runs  (si,  and  S3)  using  the  following 
formula:  s  =  ^(si^  +  S2^  +  832)73. 

The  performance  of  all  the  subjects  with  the  softest  specimen  when  using  the  right  hand, 
with  and  without  visual  force  feedback  are  listed  in  Tables  3.4  and  3.5.  With  visual  force 
feedback,  the  mean  absolute  error  of  the  three  subjects  did  not  differ  much  given  the  large 
variations  in  the  standard  deviation.  However,  without  visual  feedback,  the  performance  of 
subject  CT  is  seen  to  deteriorate  more  than  the  other  two  subjects  in  most  of  the  conditions. 

3.4.3  Statistical  tests  on  experimental  parameters 

The  analysis  of  variance  (ANOVA)  was  used  to  find  out  whether  the  force  control  perform¬ 
ance  was  influenced  by  parameters  such  as  finger  contact  conditions,  specimen  compliance, 
target  force  magnitude,  visual  feedback  conditions,  and  the  hands  used.  Data  collected  using 
different  parameters  were  grouped  together  whenever  possible  for  more  reliable  tests.  The 
complete  analysis  is  presented  in  Appendix  F  and  the  main  results  are  summarized  here. 

The  effects  of  both  specimen  softness  and  finger  contact  conditions  were  examined  together 
by  using  two-way  ANOVA  table.  The  data  for  the  three  target  force  magnitudes  were  grouped 
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Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

S.D. 

Mean 

S.D. 

S.D. 

Mean 

S.D. 

Normal 

Iggil 

0.043 

0.066 

0.028 

iiiiira 

Normal 

0.060 

0.079 

ESQ 

Normal 

8N 

QQigI 

0.078 

Em 

0.062 

0.171 

0.098 

Finger  cot 

0.036 

0.032 

0.030 

0.019 

0.024 

0.021 

Finger  cot 

0.071 

0.053 

0.171 

0.149 

0.065 

0.045 

0.051 

1 

0.039 

Finger  cot 

8N 

0.188 

0.114 

0.099 

■imlai 

esq 

iSSI 

Thimble 

mm 

llllgll 

lilillil 

iiKiW:! 

EESS 

0.025 

0.016 

Thimble 

mm 

0.072 

0.048 

0.033 

EEQ 

Thimble 

SN 

0.139 

0.152 

0.136 

0.093 

0.080 

Table  3.2:  .Absolute  error  in  force  control  for  subject  CT  using  right  hand  index  finger  with 
visual  feedback. 


Condition 

i 

Force 

Absolute  Error  (N) 

Rigid  ] 

!  Hard 

Medium 

Soft 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D.  i 

Normal  ■ 

mM 

■IKIKM 

0.029 

0.024 

0.019 

0.037 

0.028 

lilWifefel 

Normal 

mm 

0.067 

0.046 

0.049 

0.038 

esq 

esq 

Normal 

8N 

0.140 

0.114 

0.114 

0.090 

ESSI 

iiWiiki 

Finger  cot 

2N 

0.027 

0.022 

0.046 

0.037 

esq 

4N 

0.068 

0.063 

0.066 

0.054 

Finger  cot 

8N 

0.121 

0.099 

0.125 

0.089 

Thimble 

2N 

0.044  1 

0.039 

0.047 

Thimble 

4N 

ItliEfSl 

0.046 

0.039  1 

0.032 

0.061  ! 

0.048 

0.066  1 

0.049 

Thimble 

8N 

0.097 

0.081 

0.094 

0.073  ! 

0.116 

0.088 

0.103 

0.080 

Table  3.3:  Absolute  error  in  force  control  for  subject  JK  using  right  hand  index  finger  with 
visual  feedback. 
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Absolute  Error  (N) 

Subject:CH 

Subject:CT 

Subject:.JK 

S.D. 

Mean 

S.D. 

2N 

ItTifelil 

0.031 

0.021 

0.016 

0.029 

0.023 

Normal 

4N 

■[IQQ 

0.051 

iBI 

0.0.58 

0.048 

Normal 

8N 

0.073 

DBI 

0.098 

0.134 

0.103 

Finger  cot 

2N 

||2i^ 

0.035 

KiWiMni 

0.037 

■IKIhT 

0.039 

0.066 

0.0.54 

Finger  cot 

0.160 

0.129 

0.122 

0.073 

OBI 

0.104 

0.041 

0.025 

0.016 

|iBi 

0.047 

Thimble 

4N 

0.030 

0.042 

0.024 

0.066 

0.049 

Thimble 

8N 

0.077 

0.137 

0.080 

0.103 

0.080 

Table  3.4:  Absolute  error  in  force  control  for  all  subjects  using  right  hand  index  finger  with 
visual  feedback  on  the  softest  specimen. 


Condition 

F'orce 

Absolute  Error  (N) 

Subject:CH 

Subject  :CT 

Subject:JK 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N  ■ 

0.123 

0.069 

0.184 

0.139 

0.065 

0.034 

Normal 

4N 

0.135 

0.109 

0.509 

0.263 

0.123 

0.085 

Normal 

8N 

0.164 

0.132 

0.875  • 

0.461 

0.475 

0.260 

Finger  cot 

2N 

0.157 

0.119 

0.202 

0.113 

0.117 

0.098 

Finger  cot 

4N 

nnjQga 

0.224 

0.204 

0.132 

Finger  cot 

8N 

0.265 

0.642 

0.331 

Thimble 

2N 

0.214 

0.091 

0.127 

0.083 

0.125 

0.121 

Thimble 

4N 

0.104 

0.095 

0.532 

0.335 

0.094 

0.062 

Thimble 

8N 

0.833 

Table  3.5:  Absolute  error  in  force  control  for  all  subjects  using  right  hand  index  finger  without 
visual  feedback  on  the  softest  specimen. 
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together  in  this  analysis.  To  minimize  the  effect  of  target  force  in  the  analysis,  the  performance 
data  was  expressed  in  terms  of  percentage  of  the  target.  Twelve  ANOVA  tables  representing 
data  from  different  subjects,  visual  feedback  conditions  and  hands  were  constructed.  Based 
on  results  of  the  overwhelming  majority  of  the  ANOVA  tables  (11/12),  the  likelihood  that 
the  specimen  softness  and  tlie  finger  contact  conditions  played  important  roles  in  these  tasks 
was  small. 

One-way  ANOV.A  tables  were  constructed  to  compare  target  force  magnitude  factor.  The 
columns  stand  for  the  data  collected  at  three  target  force  levels  under  various  specimen  and 
contact  interface  conditions.  For  all  subjects,  there  were  significant  differences  between  the 
errors  for  2N,  4N,  and  8N  target  force  magnitudes.  For  subject  CH,  the  main  difference  came 
from  the  data  for  8N  target  force.  For  subjects  CT  and  JK,  the  differences  between  2N  and 
4N  target  force  magnitudes  were  big  enough  to  reject  the  hypothesis  that  their  performance 
were  the  same  at  1%  significance  level  under  both  visual  force  feedback  conditions.  ANOVA 
tables  using  the  errors  e.xpressed  as  a  percentage  of  target  force  were  also  constructed.  With 
visual  force  feedback  for  both  subjects  CH  and  .IK,  the  percentage  errors  were  significantly 
lower  when  controlling  4N  and  8N  target  force  than  in  controlling  2N  target  force.  For 
subject  CT,  the  percentage  errors  were  not  significantly  different  under  the  three  different 
target  force  levels  with  visual  feedback.  However,  without  visual  feedback,  the  percentage 
error  was  significantly  higher  for  subject  CT  at  8N  target  force. 

To  determine  the  effect  of  visual  feedback,  one-way  ANOVA  tables  were  constructed  by 
using  the  performance  data,  expressed  as  a  percentage  of  target  force,  collected  from  each 
subject  by  pooling  data  from  various  finger  conditions,  target  force  magnitude,  and  specimen 
softness.  The  results  rejected  the  null  hypothesis  for  all  three  subjects  with  very  high  F  ratios 
at  1%  significance  level  and  found  significant  differences  between  the  performances  with  and 
without  visual  force  feedback. 

To  compare  the  factor  of  hand  used,  one-way  ANOVA  tables  were  constructed  by  using 
the  performance  data,  expressed  as  a  percentage  of  target  force,  collected  from  each  subject 
by  pooling  data  from  various  finger  conditions,  target  forces,  and  specimen  softness.  The 
results  indicate  that  there  was  a  significant  difference  between  the  performance  of  the  two 
hands  of  subject  JK  in  controlling  forces  both  with  or  without  visual  feedback.  For  subject 
CT,  there  was  a  significant  difference  between  the  performances  with  the  two  hands  only 
when  the  visual  feedback  was  not  available.  As  far  as  subject  CH  is  concerned,  no  significant 
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difference  in  performance  was  found  between  the  two  hands. 


3.5  Discussion 

3.5.1  Effects  of  various  experimental  parameters 

The  force  control  experiments  conducted  had  parameters  including  the  specimen  softness, 
finger  contact  conditions,  the  target  force  magnitude,  the  hands  used,  and  whether  or  not 
the  force  exerted  was  visually  fed  back  to  the  subjects.  From  the  data  collected,  not  all 
the  parameters  had  significant  effects  on  the  controlling  of  normal  contact  force  with  the 
specimens. 

The  statistical  tests  conducted  indicate  that  the  finger  contact  condition  as  well  as  the 
specimen  softness  did  not  have  significant  effects  on  the  results.  The  target  force  magnitude 
did  have  significant  effects  when  the  mean  absolute  error  in  force  was  used  in  the  tests.  The 
percentage  error  was  also  used  in  the  statistical  tests  as  the  performance  index  to  normalize 
the  error.  Both  subjects  CH  and  JK  had  significantly  lower  percentage  error  controlling  4N 
and  8N  targets.  For  subject  CT,  the  difference  was  less  significant  among  all  three  target 
forces.  Not  surprisingl}^,  when  there  was  visual  force  feedback,  the  error  was  smaller  than 
without  feedback  for  all  the  subjects.  For  both  subject  CH  and  JK,  the  percentage  error 
increased  from  less  than  2%  at  8N  target  with  visual  force  feedback  to  about  6%  and  4% 
without  visual  force  feedback,  respectively.  In  the  case  of  subject  CT,  the  increase,  from  2% 
to  11%  at  8N,  was  more  dramatic. 

The  tests  on  the  hands  used  show  that  there  were  significant  differences  between  the 
results  obtained  from  two  hands  of  subject  JK,  whether  or  not  visual  force  feedback  was 
present.  For  subject  CT,  significant  difference  between  the  performance  with  the  two  hands 
was  found  only  when  visual  force  feedback  was  not  present.  But  the  tests  on  subject  CH  were 
not  able  to  reject  the  hypothesis  that  the  performances  with  the  two  hands  was  the  same. 

In  the  experiments  conducted  by  Srinivasan  and  Chen  [39],  the  mean  absolute  error  in 
tracking  constant  normal  forces  up  to  1.6N  was  about  0.04N.  The  results  obtained  from  this 
study  were  similar  to  that  result  at  2N  (see  Table  3.4)  for  not  only  the  rigid  specimen  but 
also  for  the  compliant  specimens.  However,  when  the  target  force  magnitude  increased  to  4N 
and  8N,  the  mean  absolute  error  increased  for  each  subject  but  the  mean  percentage  error 
(about  2%)  did  not  differ  significantly  for  all  the  subjects. 
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3.5.2  Force  control  in  haptic  interaction  with  objects 

The  objects  that  we  interact  with  as  well  as  the  contact  interface  conditions  can  vary  quite 
a  lot  from  one  haptic  task  to  another.  From  the  results  obtained  in  this  study,  we  know  that 
our  abilities  in  controlling  normal  forces  of  contact  with  objects  of  various  softnesses  under 
different  contact  conditions  did  not  differ  much  across  the  target  forces  tested  (2N,  4N,  and 
8N). 

The  major  differences  in  the  abilities  of  subjects  in  terms  of  controlling  contact  forces 
under  various  parameters  seemed  more  obvious  when  there  was  no  visual  force  feedback.  As 
ol:iserved  from  the  force  data  of  the  experimental  runs  (from  10  to  1-5  seconds)  which  did  not 
have  visual  force  feedback,  subject  CT  had  more  error  in  controlling  the  applied  force  than 
the  other  two  subjects,  even  though  this  subject  was  as  capable  in  controlling  force  as  the 
other  two  during  visual  feedback  stage.  The  possible  explanations  include;  (1)  the  subject 
was  not  able  to  sense  as  well,  but  because  of  the  visual  feedback  available  at  the  second  stage, 
the  suljject  performed  as  well  as  the  other  two  during  that  stage;  or  (2)  the  subject  was  not 
able  to  remember  the  forces  as  well  as  the  other  two  subjects;  thus  maintaining  a  contact  force 
without  visual  feedback  was  difficult.  In  a  general  haptic  discrimination  task  which  typically 
involves  force  control  without  visual  force  feedback,  one  might  expect  that  the  subjects  who 
cannot  control  well  in  the  tasks  (as  experimented  here)  would  not  perform  as  well  as  other 
subjects.  The  softness  discrimination  experiments  presented  in  the  previous  chapter  shows 
that  the  subject  whose  force  control  (without  visual  feedback)  performance  was  not  as  good 
as  that  of  the  other  two  subjects  did  not  perform  as  well  in  the  discrimination  tasks  also. 

We  interact  with  objects  daily  using  both  of  our  hands.  The  abilities  of  our  finger  force 
control  from  the  two  hands  may  differ  just  as  any  other  sense.  It  was  found  from  the  statistical 
tests  conducted  on  the  results  of  these  experiments  that  there  were  significant  differences 
between  the  two  index  fingers  of  subject  JK  in  controlling  normal  forces  of  contact.  As 
pointed  out  in  the  Discussion  section  oii  softness  discrimination,  the  forces  used  by  the 
two  fingers  on  the  pair  of  specimens  were  quite  different  for  subject  JK,  which  is  again 
very  consistent  with  the  findings  here.  Although  such  differences  exist,  from  the  excellent 
discrimination  performance  of  JK,  it  seems  that  our  haptic  system  can  compensate  for  it. 
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Chapter  4 


Contact  Visualization 

4.1  Abstract 

When  a  finger  contacts  an  object,  the  extend  of  the  contact  region  depends  on  the  force 
applied.  As  the  force  increases,  the  contact  grows  and  the  mechanoreceptors  under  the  skin 
surface  respond  and  convey  information  about  the  contact  through  the  nerve  pathways  to  the 
central  nervous  sj'stem.  Contact  visualization  is  a  step  towards  understanding  the  process 
of  contact  development.  A  real-time  imaging  setup  which  consisted  of  a  videomicroscopy 
system  and  a  tactile  stimulator  system  was  develo]>ed  to  observe  the  region  of  contact  and 
to  perform  controlled  experiments  by  using  transparent  objects.  When  the  finger  contacted 
the  transparent  specimens,  the  contact  developed  gradualljc  Initially,  there  were  islands  of 
contact.  As  the  contact  force  increased,  the  islands  got  larger  and  then  connected  to  form 
finger  ridge  patterns.  However,  not  all  the  regions  inside  the  overall  contact  boundary  were  in 
contact  even  at  high  forces.  In  order  to  analyze  the  contact  images,  \'arious  image  processing 
techniques  were  applied  and  developed  to  improve  the  contact  images  and  to  determine  the 
contact  as  distinct  from  non-contact  regions  in  order  to  extract  contact  information.  Vari¬ 
ous  contact  variables,  including  nominal  area,  actual  area,  percentage  contact  area,  nominal 
mean  pressure,  and  actual  mean  pressure,  were  calculated  by  using  data  obtained  from  the 
active  contact  of  the  fingerpad  with  a  rigid  Plexiglass  plate.  During  indentation,  the  nom¬ 
inal  area  saturated  quickly  while  the  actual  area  inside  the  overall  contact  region  increased 
gradually.  During  retraction,  both  the  nominal  area  and  the  actual  area  decreased  slowly. 
On  the  other  hand,  the  variations  of  the  nominal  mean  contact  pressure  were  gradual  during 
both  indentation  and  retraction.  Surprisingly,  there  was  almost  a  step  change  in  the  actual 
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mean  contact  pressure  at  the  beginning  of  contact,  after  which  it  was  remained  almost  con¬ 
stant  during  indentation  and  reduced  gradually  during  retraction.  The  percentage  contact 
area,  obtained  by  dividing  the  actual  area  by  the  nominal  area,  increased  gradually  during 
indentation  but  remained  almost  constant  during  retraction.  When  soft  specimens  of  varying 
compliance  were  used,  the  nominal  contact  area  observed  using  low  magnification  was  about 
the  same  for  all  tlie  specimens  when  the  forces  were  less  than  IN.  At  high  contact  forces,  low 
magnification  settings  used  to  record  the  entire  contact  region  resulted  in  poor  image  quality 
due  to  the  deformation  of  the  specimens.  When  high  magnifications  were  used  to  capture 
only  portions  of  the  contact  region,  the  percentage  contact  areas  for  the  softer  specimens  were 
found  to  be  higher  than  those  for  harder  specimens.  From  the  visualization  of  the  fingerpad 
actively  slipping  against  a.  flat  Plexiglas  plat,  it  was  observed  that  the  motions  of  the  finger 
ridges  near  the  boundaries  of  contact  moved  relative  to  the  plate,  which  caused  the  finger 
ridges  to  stretch  at  the  leading  edge  and  compress  at  the  trailing  edge.  These  relative  move¬ 
ments  of  the  finger  ridges  will  most  likely  cause  the  mechanoreceptors  underneath  the  skin 
to  respond  and,  through  the  central  nervous  system,  convey  to  the  brain  information  about 
the  incipience  of  slip.  The  brain  can  then  activate  the  motor  subsystem  to  take  appropriate 
actions  to  avoid  objects  from  slipping  through  our  fingers.  One  reasonable  hypothesis  for  the 
boundaries  to  slip  first  is  that  the  pressure  distribution  on  the  contact  region  is  in  fact  not 
uniform.  If  the  contact  pressure  in  the  middle  was  higher  and  decreased  until  it  reached  the 
boundary  of  contact,  the  observed  slip  phenomenon  can  be  explained  in  terms  of  Coulomb 
friction.  The  visualization  system  and  the  image  processing  techniques  developed  in  this 
study  can  be  applied  to  other  biomechanical  experiments  involving  fingers,  such  as  for  the 
automated  fingerprint  identification  system  involving  image  capture,  enhancement,  and  data 
compression. 

4.2  Introduction 

In  studying  various  aspects  of  the  human  sense  of  touch,  it  is  important  to  understand  the 
biomechanics  of  contact  during  manual  interactions  with  objects.  During  the  process  of 
finger-object  contact  for  exploration  or  manipulation,  the  mechanoreceptors  underneath  the 
skin  are  an  important  source  of  information.  Since  these  sensors  are  embedded  underneath  the 
finger  ridges,  visualizing  the  contact  region  will  help  us  understand  what  kind  of  mechanistic 
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information  is  available  to  the  tactile  sensors.  As  the  finger  comes  in  contact  with  an  object, 
the  contact  region  develops  gradually.  The  overall  area  as  well  as  the  details  inside  the 
contact  boundary  vary  with  properties  of  the  object,  properties  of  the  finger,  and  the  forces 
of  interaction.  Therefore,  the  objective  of  this  study  were  to  develop  systems  to  enable 
visualization  of  the  contact  region  and  to  find  out  how  contact  information  such  as  contact 
forces,  the  overall  contact  area,  and  the  percentage  contact  area  change  during  the  course  of 
contact. 

.4fter  contact  of  the  fingerpad  with  an  object,  a  fingerprint  is  often  left  on  the  surface  of  the 
object.  It  is  not  too  difficult  to  obtain  a  latent  fingerprint  left  behind  by  fingers  on  a  variety 
of  objects  [19]  [14].  In  general,  for  latent  fingerprinting,  one  will  need  tools  and  materials 
such  as  some  talcum  powder,  a  fine  paintbrush,  clear  cellophane  tape,  and  a  shiny  black 
paper.  By  dusting  the  object  with  talcum  powder,  blowing  on  the  talcum  |)ower  gently,  and 
Ijrushing  the  powdered  spots  lightly,  a  pattern  will  show  and  cellophane  tape  can  be  used  to 
lift  it  up  and  put  it  on  the  shiny  black  paper  for  examination.  Although  it  is  not  too  difficult, 
one  will  have  to  practice  this  a  few  times  till  one  gets  the  hang  of  it.  This  simple  procedure 
takes  a  lot  of  time  and  is  only  good  for  latent  fingerprints  and  is  not  suitable  for  dynamic  and 
real-time  observations  which  are  necessary  to  capture  the  progress  of  contact.  To  find  out 
the  contact  area,  ink  fingerprinting  would  work  just  fine  for  some  applications  that  requires 
the  overall  contact  area.  For  a  more  detailed  level  of  study  on  the  contact  region,  however, 
a  more  capable  system  is  called  for.  Thus  a  videomicroscopy  system  which  can  achieve  such 
goals  was  developed.  In  addition,  to  perform  controlled  contact  tasks,  a  tactile  stimulator 
system  was  also  developed,  which  was  used  to  control  the  motion  of  a  transparent  object  to 
indent  a  finger  which  is  stationary  (passive  touch).  With  the  two  systems  integrated  together, 
the  development  of  the  contact  region  between  the  fingerpad  and  objects  can  be  examined  in 
a  controlled  manner  in  real-time. 

Studies  on  various  aspects  of  haptic  interactions  have  utilized  the  information  obtained 
from  the  contact  region.  For  example,  contact  area  estimation  was  used  to  relate  to  the 
perceived  curvature  in  the  research  conducted  by  Goodwin  et  al  [18]  on  the  tactile  dis¬ 
crimination  of  curvature.  They  estimated  the  area  of  contact  between  skin  and  objects  with 
different  curvature  by  ink  fingerprinting.  They  found  that  as  the  curvature  increased,  the  con¬ 
tact  area  decreased.  And  lower  contact  force  resulted  in  a  decrease  in  contact  area.  In  the 
study  on  human  discrimination  of  thickness.  Ho  [7]  estimated  the  contact  area  between  finger 
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and  plates  of  various  thickness  by  ink  fingerprinting  and  the  data  was  used  in  finite  element 
analysis  to  compute  the  deformed  plate  curvature.  By  using  the  videomicroscopy  system 
developed  in  this  study,  Dandekar  [10]  conducted  experiments  to  perform  3D  reconstruction 
of  the  primate  fingerpads  for  finite  element  analysis  and  also  measured  surface  deformation 
of  human  fingertip  in  vivo  with  indenters  of  various  sizes  and  shapes. 

In  this  study,  the  focus  was  on  the  visualization  of  the  contact  region  with  objects  of  dif¬ 
fering  com]>!iance.  This  chapter  includes  descriptions  of  the  various  devices  developed  which 
enable  such  observations,  image  processing  techniques  applied  to  enhance  or  visualize  the 
image  data,  observations  of  the  slip  phenomenon  between  finger  and  rigid  Plexiglas,  and  also 
presents  data  obtained  from  controlled  contact  experiments  with  soft  objects.  The  visualiz¬ 
ation  systems  develo|)ed  in  the  study,  the  accompanying  image  processing  and  visualization 
software,  and  the  results  on  contact  region  development  can  be  applied  to  other  biomechanical 
experiments  involving  fingers  and  also  to  the  automated  fingerprint  identification  systems  [19] 
for  image  capture,  enhancement,  and  data  compression. 

4.3  Methods 

4.3.1  Videomicroscopy  system 

A  videomicroscopy  system  capable  of  real-time  imaging  was  developed  to  observe  the  region 
of  the  human  finger  in  contact  with  transparent  objects.  Figure  4-1  is  a  schematic  diagram 
of  the  system  which  includes  a  fiber  optic  light  source,  a  gray  scale  CCD  camera,  a  zoom 
lens,  a  video  monitor,  and  a  frame  grabber  hosted  by  a  80486  personal  computer. 

The  contact  region  is  illuminated  by  a  Fostec  Model  8375  fiber  optic  light  source.  For 
high  magnification  images,  the  light  illuminates  the  contact  region  coaxially  through  the  beam 
splitter  and  zoom  lens.  For  low  magnification  images,  an  alternative  light  is  used  as  indicated 
in  the  figure.  The  Unitron  1:6.5  zoom  lens  provides  a  range  of  magnifications  of  the  contact 
region,  enabling  the  overall  contact  area  as  well  as  the  details  of  islands  of  contact  with  a  few 
finger  ridges  to  be  imaged.  The  output  of  the  high  resolution  Hitachi  Model  KP-MlU  CCD 
camera  is  displayed  on  a  Hitachi  video  monitor.  The  camera  output  can  also  be  digitized  by 
a  BitFlow  VESA  local-bus  VideoRaptor  frame  grabber  in  the  80486  personal  computer. 

Using  this  videomicroscopy  system,  video  data  can  be  digitized  in  real-time  and  trans¬ 
ferred  from  the  frame  grabber  to  the  computer  system  memory.  The  amount  of  digitized 
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Figure  4-1:  The  videomicroscopy  system. 

video  data  is  thus  limited  to  the  available  memory  in  the  computer  system.  For  a  typical 
640.\480  8-bit  gray  scale  image  resolution,  the  16MB  memory  inside  the  computer  can  hold 
about  .54  frames  of  video  which  amounts  to  about  1.8  seconds  of  data  if  digitized  at  a  rate  of 
30  frames  per  second.  Observations  made  from  daily  contact  tasks  suggest  that  the  develop¬ 
ment  of  a  typical  finger-object  contact  usually  has  a  time  span  of  less  than  2  seconds.  For 
slower  contact  speed  (the  contact  region  will  also  vary  slowly),  the  video  acquisition  rate  can 
be  reduced  to  accommodate  a  longer  period  of  contact  without  sacrificing  either  the  duration 
or  the  accuracy  of  the  analysis. 

4.3.2  Tactile  stimulator  system 

A  tactile  stimulator  was  developed  to  investigate  the  contact  phenomena  in  a  controlled  man¬ 
ner.  Figure  4-2  is  a  diagram  of  the  tactile  stimulator  system.  This  linear  stepper  motor  based 
tactile  stimulator  is  used  to  carry  a  specimen  to  indent  a  stationary  finger  at  a  given  velocity. 
The  Industrial  Device  Corporation  (IDC)  NS2T205A-2  linear  electric  cylinder  together  with 
a  IDC  S5101  micro-stepping  drive  has  a  step  size  of  less  than  1  micrometer  which  makes  it 
easy  to  achieve  high  precision. 

As  indicated  in  Figure  4-2,  the  motor  is  controlled  by  a  80386  personal  computer  with 
a  custom  digital  input/output  interface.  By  changing  the  frequency  of  the  step  pulses  sent 
to  the  motor,  the  speed  of  indentation  can  be  accurately  controlled.  The  timing  of  the  step 
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Specimen 


Figure  4-2:  The  tactile  stimulator  system. 


frequency  is  controlled  via  a  IMHz  custom  timing  interface. 

In  order  to  observe  the  contact  forces  between  the  specimen  and  the  human  finger,  a 
strain  gage  based  single  degree  of  freedom  force  sensor  was  built  and  mounted  between  the 
specimen  holder  and  the  motor  cylinder.  Binocular  spring  element  was  used  to  construct 
the  sensor  because  it  is  more  rigid  and  less  sensitive  to  location  of  force  application.  Four 
strain  gages,  made  by  Measurements  Group,  were  mounted  on  the  top  and  bottom  surfaces 
of  the  force  sensor  near  the  center  of  the  two  holes  of  the  binocular  to  form  a  full  bridge.  The 
force  signal  was  amplified  by  a  signal  conditioner  before  it  was  digitized  by  a  data  acqusition 
system  in  a  computer. 

4.3.3  Real-time  imaging  setup 

Both  the  videomicroscopy  and  the  tactile  stimulator  systems  can  be  used  by  themselves  to 
collect  useful  data.  For  example,  the  videomicroscopy  system  was  used  for  the  3D  reconstruc¬ 
tion  of  the  fingerpad  and  observe  the  surface  deformation  of  the  fingertips  under  a  variety  of 
indentors  [10].  The  integration  of  the  two  systems  which  would  provide  synchronized  force 
and  video  data  collection  is  one  of  the  important  goals  of  this  research. 

Since  digitizing  the  video  data  and  controlling  the  stimulator  both  require  constant  at- 
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tention  of  the  central  processing  unit  of  the  computer  (CPU),  two  computers  were  used  to 
meet  the  two  needs.  In  such  a  design,  the  synchronization  of  the  two  tasks  was  a  major 
concern.  This  problem  can  be  solved  by  having  an  accurate  custom  timer  interface  on  both 
computers  and  use  digital  input/output  interfaces  to  allow  communication  between  them  for 
synchronization.  Figure  4-3  shows  a  schematic  of  the  integrated  system.  The  communica¬ 
tion  between  the  two  computers  is  achieved  by  sending  command  data  through  the  custom 
digital  input/output  interfaces  installed  on  both  computers.  The  80386  PC  is  dedicated  to 
the  motion  control  of  the  stimulator  while  the  80486  PC  records  both  video  and  force  data 
during  contact.  The  80386  PC  receives  parameters  such  as  motor  speed  and  travel  distance 
from  the  80486  PC  and  executes  the  command  accuratel\',  by  using  the  IMHz  timer,  after 
receiving  the  start  signal.  On  the  other  hand,  the  80486  PC  coordinates  the  experiment  and 
collects  video  and  force  data  according  to  its  own  time  base  which  is  the  same  custom  timing 
interface  as  the  one  in  the  80386  PC. 

.4s  a  result  of  this  integration,  the  two  computer  systems  work  together  with  their  own 
control  software  and  the  setup  is  capable  of  a  video  rate  of  at  least  20  frames  per  second  and 
a  force  rate  of  1000  samples  per  second.  However,  such  high  data  rate  can  only  be  sustained 
for  a  short  period  of  time,  usually  a  little  less  than  2  seconds,  because  of  the  amount  of 
system  memory  available  on  the  80486  computer  to  store  the  digitized  image  (about  300 
KByte  per  frame).  Typical  contact  duration  of  tasks  such  as  softness  discrimination  takes 
about  1  second.  Therefore,  the  2  second  capture  time  allowed  by  this  setup  is  sufficient  for 
achieving  the  goals  of  the  investigations. 

4.3.4  Active  and  passive  touch  experiments 

Performing  active  and  passive  touch  experiments  to  obtain  contact  information  is  a  crucial 
step  towards  the  understanding  of  the  mechanical  inputs  to  the  mechanoreceptors  under  the 
skin.  The  real-time  imaging  setup  (Figure  4-3)  was  used  to  visualize  the  contact  region  and 
record  the  contact  forces.  In  passive  touch  experiments,  the  finger  was  attached  to  a  stationary 
finger  holder  and  the  tactile  stimulator  applied  specimens  to  the  finger  at  controlled  velocity 
of  2.4mm/sec.  During  active  touch  experiments,  the  subject’s  finger  was  not  constrained. 
Instead  of  moving  the  tactile  stimulator,  the  subjects  moved  their  finger  to  touch  the  stationary 
specimen. 

Transparent  rigid  Plexiglas  and  compliant  silicon  rubber  specimens  were  used  in  the 
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Specimen 

Figure  4-3:  The  real-time  imaging  setup. 


experiments.  By  adjusting  the  amount  of  diluent  GE  SF96-50  mixed  to  fixed  proportions 
of  GE  RTV615A  and  RTV615B  silicon  rubber  components,  the  compliance  of  the  specimen 
can  be  varied.  The  procedure  and  method  used  to  cast  and  calibrate  the  specimens  are 
described  in  detail  in  the  Methods  section  on  softness  discrimination  experiments.  Three  soft 
specimens  with  objective  compliance  value  of  0.086mm/N  (specimen  code  AB4),  0.188mm/N 
(code  BA4),  and  0.304mni/N  (code  CA4)  were  used.  The  transparent  rigid  Plexiglas  used 
had  a  thickness  of  1/4  inch. 

The  custom-made  binocular  force  sensor  was  used  to  record  the  contact  force.  The 
sensor  has  a  range  of  40N  and  the  data  is  digitized  bj'  a  12-bit  data  acquisition  board  at 
the  rate  of  1000  samples  per  second.  Due  to  its  binocular  design,  the  sensor  is  not  very 
sensitive  to  the  location  of  load  application.  The  contact  region  was  captured  into  the  80486 
PC  system  memory  at  a  rate  of  20  frames  per  second.  In  passive  touch  experiments,  the 
recording  was  carried  out  for  the  indentation  portion.  In  the  active  touch  experiments,  the 
recording  time  span  was  about  2  seconds  and  required  coordination  between  the  subjects  and 
the  experimenter  for  a  good  capture.  Three  repetitions  were  performed  for  the  passive  touch 
experiments;  the  active  touch  experiments  were  repeated  until  three  satisfactory  sequences 
were  captured. 
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4.3.5  Observations  of  slip 

In  addition  to  the  recording  of  the  contact  region  during  passive  indentation  and  active 
touch  experiments  with  compliant  specimens,  the  videomicroscop3f  system  can  also  be  used 
to  visualize  a  variety  of  contact  related  phenomena.  For  example,  the  study  of  slip  requires 
visualization  at  the  finger  ridge  level. 

For  observations  on  slip,  the  subject  first  contacted  the  rigid  Plexiglas  and  then  stroked 
it  back  and  forth  with  the  fingerpad  in  the  horizontally  direction.  The  subject  repeatedly 
performed  the  actions  and  a  two  second  capture  was  initialized  at  the  appropriate  time  to 
record  a  part  of  the  process.  Due  to  the  setup  limitations,  only  active  touch  experiments  on 
slip  are  possible.  This  is  because,  to  have  controlled  passive  slip,  another  motor  to  control 
a  second  degree  of  freedom  is  required.  Also,  a  more  quantitative  stud,v  of  slip  requires 
shear  force  measurements.  In  the  experiments  described  here,  the  phenomenon  of  slip  was 
oljserved  and  the  sequence  of  images  obtained  was  analyzed.  Even  with  limitation  of  the 
available  setup,  the  understanding  of  slip  at  the  ridge  level  is  enhanced  by  these  observations. 
It  also  provided  good  evidence  on  the  nature  of  force  distribution  over  the  contact  region. 

4.4  Contact  image  processing 

In  the  context  of  tactile  sense,  the  portion  of  the  fingerpad  that  is  in  contact  with  the  object 
is  especially  important  since  the  mechanoreceptors  under  that  region  will  generate  most  of 
the  nerve  impulses  transmitted  to  the  central  nervous  system.  The  images  acquired  from  the 
videomicroscopy  system  are  of  high  enough  contrast  as  to  be  visible  to  the  human  eye  when 
appropriate  lighting  is  used.  In  order  to  provide  qualitative  visualization  and  to  perform 
quantitative  analysis  of  the  images,  algorithms  for  enhancing  and  extracting  the  contact 
regions  were  developed.  This  section  is  devoted  to  the  techniques  used  in  this  study  in  order, 
to  enhance  contact  images,  separate  the  contact  area  from  the  non-contact  area,  and  visualize 
various  contact  phenomena. 

4.4.1  Simple  thresholding 

When  appropriate  lighting  is  used,  the  regions  of  the  finger  that  are  in  contact  with  the 
specimen  have  lower  image  intensity  than  the  regions  that  are  not  in  contact.  This  suggests 
the  use  of  a  simple  thresholding  of  the  image  intensity  to  determine  the  contact  area.  The 
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Figure  4-4:  Application  of  simple  thresholding  to  obtain  the  contact  area- 


threshold  value  can  be  picked  out  from  the  image  histogram,  which  is  the  distribution  of  gray 
levels  in  an  image.  A  higher  gray  scale  value  in  the  distribution  corresponds  to  a  brighter 
pixel.  If  the  pixel  at  a  particular  location  has  a  lower  gray  level  than  the  threshold,  it  is  in 
contact.  Otherwise,  it  is  not. 

A  typical  result  of  applying  such  a  technique  to  obtain  the  contact  area  is  shown  in 
Figure  4-4.  The  upper  left  graph  shows  a  contact  image  of  a  fingerpad  with  rigid  Plexiglas. 
The  histogram  of  the  image  is  shown  at  the  upper  right.  The  simple  thresholding  method 
requires  one  to  pick  a  gray  scale  value  to  convert  the  original  image  into  a  binary  image 
consisting  only  of  contaot  and  non-contact  pixels.  Although  human  eyes  can  easily  pick  out 
the  contact  region  in  the  image  because  of  our  sophisticated  visual  system,  it  is  difficult 
to  pick  a  gray  level  from  the  image  histogram  to  perform  the  thresholding  operation.  The 
bottom  two  graphs  show  the  result  of  using  two  different  threshold  values.  A  threshold  value 
of  90  seems  too  low  because  it  cannot  convert  the  middle  region  into  a  binary  image  properl}'. 
If  the  threshold  value  is  increased  to  a  higher  value  of  100  in  order  to  convert  the  center  part 
properly,  the  boundaries  become  too  dark. 
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By  taking  a  closer  look  at  the  original  image  and  comparing  the  gray  levels  of  the  white 
region  in  the  center  and  that  of  the  boundaries,  one  realizes  that  the  background  of  the  image 
has  non-uniform  gray  values.  Since  the  illumination  of  the  image  is  not  perfectly  uniform, 
contact  regions  in  an  area  of  high  illumination  may  have  equal  or  greater  intensity  than  non- 
contact  regions  in  the  area  of  low  illumination.  Therefore,  the  simple  operation  of  thresholding 
cannot  convert  the  pixels  into  contact  or  non-contact  regions  satisfactorily.  However,  if  the 
effect  of  non-uniform  illumination  can  be  eliminated  by  other  techniques,  then  thresholding 
can  be  applied  to  obtain  the  contact  area  reliably. 

4.4.2  Homomorphic  processing 

One  way  of  removing  the  non-uniform  lighting  is  to  apply  homomorphic  processing  [29][17] 
Homomorphic  processing  can  be  used  to  reduce  the  dynamic  range  of  the  image  and  enhance 
the  local  contrast.  The  resulting  image  has  uniform  illumination,  thereby  facilitating  the  use 
of  the  thresholding  method  to  determine  the  area  of  contact. 

The  gray  level  of  a  contact  image  /(iii,  712)  can  be  modeled  as 


f{ni,n2)  -  z(?ii,  n2)r(ni,  712),  (4.1) 

where  7(771,712,)  represents  the  illumination  of  the  image  at  location  (711,772)  and  7’(77i,  772) 
represents  the  reflectance  of  the  object.  In  this  case,  the  illumination  7(771,  ^2)  varies  slowly 
across  the  image  with  a  low  spatial  frequency  of  one  cycle  per  image.  The  reflectance  r(r7i,  772) 
contains  the  details  of  the  fingerprint  whose  fundamental  frequency  is  determined  by  the 
number  of  finger  ridges  present  in  the  image.  Since  equation  4.1  cannot  be  used  directly 
to  perform  a  Fourier  transform  operation  in  order  to  separate  the  two  components,  we  can 
convert  it  from  a  product  to  a  sum  by  taking  the  logarithm  of  equation  4.1: 

772))  =  log{i{nu  772))  +  log{r{ni,  772)).  (4.2) 

Now,  because  of  linearity,  we  can  filter  the  log  of  /(77i,  772)  to  remove  the  illumination 
effects.  After  that,  we  can  exponentiate  the  result  to  obtain  t* (771, 772).  A  block  diagram  for 
applying  homomorphic  processing  is  illustrated  in  Figure  4-5. 

^The  homomorphic  processing  technique  was  developed  with  the  help  of  Eric  Foxlin  and  Robert  Stadler 
and  the  author  wishes  to  gratefully  acknowledge  their  assistance. 
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Figure  4-5:  A  block  diagram  for  applying  homomorphic  processing  to  determine  the  contact 
area. 
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If  the  homomorphic  processing  successfully  removes  the  illumination  effects,  the  unimodal 
histogram  (Figure  4-4)  of  the  original  image  should  be  in  the  form  of  a  bimodal  histogram. 
One  mode  represents  the  finger  ridges  that  are  in  contact  and  the  other  mode  represents 
the  regions  that  are  not  in  contact.  We  can  then  choose  a  threshold  value  i>et\veen  the  two 
modes  .so  that  the  contact  areas  in  the  image  are  black  and  non-contact  areas  are  white.  .4n_y 
significant  noise  will  be  converted  into  impulsive  noise  by  the  thresholding.  .4  median  filter 
can  serve  to  remove  the  noise  before  counting  the  pixels.  The  actual  contact  area  can  be 
calculated  by  applying  a  calibration  constant  which  represents  the  area  of  a  single  pixel.  For 
high  magnification  images  which  contain  only  a  few  finger  ridges,  the  frequency  of  the  ridges 
is  closer  to  the  illumination  frequency.  In  those  cases,  either  a.  filter  with  very  sharp  cutoff 
must  be  used  or  a  decimation  on  the  images  must  be  performed  in  order  to  separate  the  two 
areas{Figure  4-5). 

The  implementation  of  the  algorithm  is  accomplished  by  using  the  software  package  MAT- 
LAB  in  conjunction  with  custom  developed  C  programs.  The  frequency  transform  method 
was  used  for  the  filter  design.  It  begins  with  the  design  of  an  optimal  1-dimensional  filter 
using  the  Pa.rks-McClellan  algorithm.  For  an  image  that  is  640  pixels  wide,  the  illumination 
frequency  is  7r/320  (one  cycle  per  image).  If  there  are  10  finger  ridges  in  the  image,  the 
fundamental  frequency  will  be  7r/32.  So,  the  filter  must  pass  7r/32  but  filter  7r/320.  The 
1-dimensional  frequency  response  is  later  converted  into  a  2-dimensional  frequency  response 
with  the  McClellan  transformation.  The  high-pass  filtering  operation  is  performed  in  the  fre¬ 
quency  domain  by  multiplying  the  Fourier  transformed  image  and  the  high-pass  filter.  Then 
the  inverse  Fourier  transform  operation  converts  the  result  back  into  the  spatial  domain. 

Figure  4-6  shows  the  same  original  image  shown  in  Figure  4-4  and  the  resulting  image 
after  homomorphic  processing,  its  histograms  and  the  thresholded  image.  In  addition  to 
checking  that  the  histogram  is  now  bimodal,  the  results  of  the  image  processing  can  also  be 
verified  by  eye  to  see  how  well  they  represent  the  true  contact  area  shown  in  the  original 
image. 

4.4.3  Scan-line  processing 

In  real-time  applications,  reducing  the  amount  of  time  required  to  process  the  image  data  is 
critical.  For  example,  for  a  robot  equipped  with  a  vision  system  or  a  high  resolution  tactile 
sensor  array  to  manipulate  an  object  like  the  humans  do,  the  recognition  time  is  critical. 
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Figure  4-6:  Apply  homomorphic  processing  to  image. 
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Scan  Line 


Figure  4-7:  The  scan-line  processing  on  a  fingerprint  line. 


The  same  is  true  for  finger  print  security  systems.  In  real-time  inspection  or  target  detection 
applications,  optical  correlators  are  typically  used  to  achieve  the  required  speed  [2] [5] [6].  If 
using  optical  correlators  is  not  practical,  scan-line  processing  seems  to  be  the  best  choice.  By 
processing  the  image  line  by  line  using  simple  operations,  it  is  possible  to  achieve  real-time 
performance. 

The  idea  of  scan-line  processing  is  generated  from  the  image  model  used  in  homomorphic 
processing,  image  calibration,  and  other  techniques  such  as  top>-hat  and  well  transform  for 
peak  and  valley  detection  [1].  In  the  case  of  top-hat  transform,  the  bright  peaks  of  an  image 
are  revealed.  The  top-hat  image  is  created  by  first  applying  an  opening  operation  which 
removes  protruding  peaks  from  the  image.  Then  a  dual-image  point  process  is  used  to 
subtract  the  top-hat  image  from  the  original  image.  The  result  is  an  image  in  which  the 
bright  peaks  appear. 

The  scan-line  processing  method  used  here  first  estimates  the  background  image  intensity 
by  bounding  the  scan-line  to  be  processed.  The  first  step  is  to  find  the  location  of  the 
maximum  intensity  value  on  the  scan-line.  Then  a  background  is  constructed  from  the  history 
of  searching  for  the  maximum  location  starting  from  both  ends  of  the  scan  line.  The  result  is 
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Image  Threshold  =  0.95 
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Figure  -1-8:  Results  of  applying  the  scan-line  processing  to  a  fingerprint  image  with  different 
threshold  \’alues. 

usually  a.  dome  shaped  background  with  steps.  The  third  step  is  to  divide  the  original  scan- 
line  by  this  estimated  background.  The  result  is  a  gray  scale  ratio  between  0  and  1.  A  high 
contra.st  image  would  result  in  scan-lines  in  which  the  contact  pixels  would  have  gray  scale 
values  closer  to  0  than  the  result  from  a  low  contrast  image.  Afterwards,  a  threshold  ratio 
can  be  used  to  convert  the  scan-line  into  binary  images.  Figure  4-7  illustrates  an  original 
scan-line,  a  constructed  background  and  their  ratio.  The  results  of  the  scan-line  processing 
and  thresholding  are  shown  in  Figure  4-8  where  three  different  threshold  percentages  values 
were  used. 

When  the  borders  contain  finger  ridges,  error  will  occur  from  the  method  outlined  above 
because  the  gray  scale  ratio  will  be  close  to  1  when  it  should  ideally  be  lower  than  1.  For 
images  containing  a  substantial  amount  of  noise,  a  low-pass  filtering  on  the  scan-line  would 
improve  the  thresholding  result.  This  technique  is  suitable  when  the  fingerprint  does  not 
occupy  the  whole  image,  as  in  the  case  of  real-time  automated  fingerprint  recognition  or 
security  systems.  It  efficiently  converts  the  gray-scale  image  into  binary  in  real  time,  thus 
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reducing  the  data  size  to  one  eighth  of  the  original.  For  most  applications,  the  size  of  the 
resulting  image  can  be  reduced  by  compressing  the  binary  image  using  algorithms  such  as 
LZW  [30].  An  image  size  reduction  of  about  30  was  achieved  in  pilot  trials  with  fingerprint 
images.  With  the  possibility  of  real-time  binarization  of  the  original  fingerprint  images,  the 
automated  fingerprint  recognition  systems  [32] [9]  could  be  speeded  up. 

4.4.4  Morphological  processing 

Image  morphology  involves  pixel  level  processing  that  helps  to  clarify  and  simplify  objects 
of  interest[l][l7].  It  can  be  applied  to  obtain  outlines  or  skeletons  of  an  object  by  emptying 
the  interior  or  reducing  the  thickne,ss  of  the  object.  For  binary  images,  binary  morphological 
processing  uses  logical  operators  such  as  AN D  and  OR  on  neighboring  pixels  to  obtain  a 
resulting  image.  In  this  study,  it  is  used  to  obtain  the  boundaries  of  the  finger  ridges  for  the 
purpose  of  visualization  of  contact  phenomena. 

Erosion  and  dilation  are  two  fundamental  morphological  operations.  A  3  x  3  mask  size 
is  usually  used  for  the  operation.  Suppose  a  binary  image  is  composed  of  black  and  white 
pixels,  with  the  object  represented  by  white  pixels.  To  erode  away  the  boundaries  of  the 
object,  the  erosion  operation  is  called  for.  If  a  pi.xel  and  its  eight  neighbors  are  all  white, 
the  corresponding  pixel  is  white  in  the  eroded  image;  otherwise,  the  corresponding  pixel  in 
the  resulting  image  is  black.  When  erosion  is  performed  on  a  pixel  inside  an  object,  it  has 
no  effect.  However,  if  the  pixel  is  on  the  boundary  of  an  object,  since  its  neighboring  pixels 
consists  of  both  object  and  background,  it  will  be  converted  into  a  background  pixel.  If  there 
is  random  noise  consisting  of  only  a  few  pixels,  they  will  be  eroded  away  also.  Dilation  does 
the  opposite  to  the  image:  a  pixel  in  the  dilated  image  is  white  if  any  of  the  pixels  defined 
by  the  mask  is  white  in  the  original  image.  It  helps  to  make  the  object  bigger  and  fill  in  tiny 
holes  inside  the  object. 

Different  combinations  of  the  erosion  and  dilation  operations  produce  different  effects. 
The  opening  operation  is  an  erosion  operation  followed  by  a  dilation  operation  to  remove 
single-pixel  noise  spikes.  The  closing  operation  is  a  dilation  operation  followed  by  an  erosion 
to  fill  small  holes  and  gaps.  When  an  eroded  image  is  subtracted  from  the  original  image 
by  using  a  dual-image  point  process,  the  result  is  the  outline  of  the  objects  in  an  image. 
Figure  4-9  shows  the  outline  of  the  fingerprint  images  created  by  the  outlining  process.  The 
image  shown  was  first  thresholded  by  scan-line  thresholding  with  a  threshold  value  of  0.95. 
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Original  image:  A  Outline  Image:  A 


Figure  4-9:  Results  of  applying  binary  morphological  processing  to  obtain  the  outline  of  the 
fingerprint  contact  regions. 
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A  dilation  followed  by  an  erosion  operation  was  used  to  reduce  the  noise.  Two  more  erosion 
operations  were  performed  and  the  resulting  image  was  subtracted  from  the  noise  reduced 
binary  image  to  create  the  outline  image. 

4.5  Implications  on  the  mechanics  of  contact 

4.5.1  Overall  (Nominal)  and  actual  contact  areas 

When  the  finger  comes  in  contact  with  objects,  the  contact  region  gradually  develops.  At 
first,  there  are  islands  of  contact  corresponding  to  contact  points  on  each  of  the  finger  ridges. 
.4s  the  normal  force  of  contact  increases,  the  islands  grow  bigger  and  the  overall  (nominal) 
contact  boundary  grows  as  well.  The  islands  of  contact  eventually  get  connect  together  result 
in  the  fingerprint  patterns.  Figure  4-10  shows  part  of  a  sequence  of  images  obtained  when 
a  fingerpad  actively  contacted  a  Plexiglas  plate.  The  image  sequence  was  recorded  at  a  rate 
of  20  frames  per  second  under  low  magnification  and  compressional  force  of  contact  was 
recorded  at  1000  samples  per  second.  Therefore,  there  were  .50  force  samples  within  a  single 
frame.  The  corresponding  mean  forces  exerted  during  an  image  frame  are  also  shown  in  the 
figure.  The  growth  of  the  overall  contact  boundary  can  be  seen  by  comparing  consecutive 
frames.  To  quantitatively  describe  the  growth  of  the  contact  region,  the  overall  area  of  the 
contact  boundary  and  the  actual  areas  inside  the  contact  region  can  be  useful  measures. 

To  obtain  the  overall  area,  the  border  of  the  image  region  occupied  by  the  fingerprint  needs 
to  be  extracted.  The  border  extraction  was  performed  in  two  steps.  First,  the  original  contact 
images  were  converted  into  binary  images  by  using  scan-line  thresholding  or  homomorphic 
processing.  Then,  an  estimate  of  the  border  was  obtained  by  using  a  technique  similar  to 
the  wedge-ring  detection  method  broadly  used  in  pattern  recognition  [9].  In  this  modified 
wedge-ring  method,  the  original  image  was  partitioned  by  using  wedges  and  rings  formed 
by  concentric  circles  and  radial  lines.  Figure  4-11  shows  the  wedge-ring  pattern  that  was 
used  to  count  the  contact  pixels  and  obtain  the  contact  region  boundary.  The  actual  pattern 
consisted  of  more  rings  and  wedges  than  in  the  one  shown.  The  numbers  were  determined 
by  using  typical  images  and  by  adjusting  on  the  basis  of  visual  inspection  of  the  extracted 
boundary  smoothness.  The  contact  pixels  from  a  binary  image  were  used  to  first  calculate 
the  approximate  center  of  the  contact  region  by  using  the  averaged  coordinates.  Then  the 
wedge-ring  pattern  centered  around  the  contact  center  was  superimposed  to  collect  contact 
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Image  2;  0.26N 


Images;  1.10N  !m.age  6: 1 .30N 


Figure  4-10:  A  sequence  of  images  of  contact  between  a  fingerpad  and  Plexiglas  specimen. 
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pixels  within  each  cell.  Once  the  pixels  were  assigned  to  the  appropriate  cells  by  determining 
their  distance  and  angles  with  respect  to  the  center  of  the  contact  region,  the  cumulative  sum 
of  contact  pixels  within  the  cells  along  each  wedge  starting  from  the  center  was  obtained. 
An  estimate  of  the  boundary  can  be  made  based  on  the  ring  radius  inside  which  most  of  the 
contact  pixels  reside.  A  typical  ratio  of  99%  was  used  to  avoid  noise  existing  outside  the 
border,  which  also  showed  up  as  contact  pixels.  The  number  of  wedges  and  rings  used  would 
roughly  determine  the  errors  in  the  boundary  estimation.  Figure  4-12  shows  a  binary  contact 
image  and  the  result  of  the  boundary  points  estimated  by  using  the  above  procedure.  The 
number  of  wedges  and  rings  used  were  48  and  80,  respectively.  In  general,  using  more  than 
30  wedges  and  60  rings  will  generate  satisfactory  estimates  for  the  border.  However,  if  the 
number  of  wedges  is  too  many,  the  resulting  boundaries  may  not  vary  smoothly. 

The  total  number  of  pixels  inside  the  boundary  were  estimated  from  the  coordinates 
of  the  border  points  and  that  of  the  contact  region  center.  Calibration  of  image  size  was 
performed  before  the  contact  images  were  obtained.  The  calibration  constants  depend  on 
the  magnification  of  the  images  and  can  be  found  by  imaging  a  reticle  which  has  calibration 
patterns  and  by  counting  the  corresponding  number  of  pixels  that  occupy  a  known  area. 
The  overall  contact  area  estimates  can  then  be  obtained  by  applying  the  calibration  constant 
which  represents  a  unit  pixel  area  for  a  particular  contact  image  sequence. 

Although  the  overall  contact  area  is  a  number  which  represents  the  nominal  size  of  the 
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Figure  4-12:  An  estimate  of  the  overall  contact  region  boundary  using  the  modified  wedge¬ 
ring  method. 

contact  region,  the  actual  area  of  contact  inside  the  contact  boundary  is  as  important.  For 
example,  given  a  contact  force,  a  nominal  mean  pressure  can  be  determined  if  the  overall 
contact  area  is  known.  However,  from  results  of  visualizing  the  contact  region,  we  know  that 
not  all  regions  inside  the  contact  boundary  are  actually  in  contact.  Therefore,  the  nominal 
mean  pressure  can  be  considerably  less  than  the  actual  mean  pressure  if  the  actual  contact 
area  is  only  a  small  portion  of  the  overall  area.  The  actual  area  in  contact  can  be  estimated  by 
counting  the  contact  pixels  after  thresholding,  although  such  estimates  may  have  inherently 
higher  errors  due  to  the  fact  that  low  magnification  was  used  to  acquire  the  overall  area.  By 
using  high  magnification  capture  of  the  central  portion  of  the  contact  images,  the  actual  area 
estimations  will  be  more  accurate  owing  to  higher  resolution.  However,  depending  on  the 
locus  of  the  image  within  the  contact  region,  the  percentage  contact  can  vary  since  the  finger 
ridge  pattern  is  different  in  different  parts  of  the  fingerpad. 

The  following  illustrates  various  pieces  of  information  available  during  contact  by  using 
data  from  active  touch  experiments  on  fingerpad  contacting  with  a  rigid  Plexiglas  plate  under 
low  magnification.  Figures  4-13  and  4-14  show  several  variables  plotted  against  time  and 
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force  for  subject  CH.  The  profiles  of  the  plotted  variables  are  divided  into  two  phases;  the 
indentation  phase  and  the  retraction  phase.  The  indentation  is  marked  with  crosses  whereas 
the  retraction  portion  is  marked  with  circles.  Since  the  first  few  frames  digitized  did  not 
contain  contact  images,  the  calculations  were  done  for  the  frames  which  had  a  mean  force 
larger  than  O.IN. 

In  Figure  4-13,  the  first  graph  shows  the  force  profile  which  reached  its  maximum  value 
just  before  the  1  second  time  mark.  The  corresponding  nominal  contact  area  increased  as  the 
force  increased  during  the  indentation  with  an  initial  rapid  increase  followed  by  a  slower  rate 
as  shown,  it  saturated  even  before  the  contact  forces  reached  the  maximum.  By  comparing 
the  nominal  area  witli  the  actual  contact  area,  it  can  be  observed  that  during  the  later  part 
of  the  indentation,  the  actual  area  kept  on  increasing  even  after  the  overall  contact  saturated. 
The  actual  area  divided  by  the  nominal  area  is  shown  as  the  percentage  of  contact  inside  the 
nominal  contact  area:  it  increased  during  the  indentation  phase  and  seemed  to  stay  constant 
in  the  figure  during  the  retraction  phase.  The  mean  nominal  contact  pressure  was  obtained 
by  dividing  the  contact  forces  by  the  nominal  contact  area.  The  shape  of  the  mean  nominal 
pressure  is  close  to  that  of  the  force  profile.  On  the  other  hand,  when  the  actual  contact  area 
was  used  for  pressure  calculations,  the  mean  actual  pressure  obtained  showed  qualitatively 
different  characteristics  from  the  mean  nominal  pressure.  Unlike  the  nominal  pressure  which 
rose  very  fast  durint  indentation,  the  increase  in  the  actual  pressure  was  slow.  The  pressure 
drop-off  rate  for  both  nominal  and  actual  was  about  the  same  during  retraction.  Since  the 
percentage  contact  inside  the  overall  area  ranged  from  10  %  at  the  beginning  to  25  %  during 
the  retraction,  the  actual  pressure  was  at  least  4  times  as  large  as  the  nominal  pressure. 
Figure  4-14  provides  a  different  view  of  the  data  obtained.  The  difference  between  the  two 
phases  of  contact  is  more  dramatic  when  plotted  against  force.  Given  the  same  amount  of 
force,  the  contact  information  strongly  depends  on  whether  it  was  in  the  indentation  phase 
or  the  retraction  phase.  The  only  variable  which  was  similar  during  the  two  phases  was  the 
nominal  contact  pressure. 

4.5.2  Contact  areas  with  soft  specimens 

The  real-time  imaging  setup  was  used  to  observe  the  contact  areas  of  the  fingerpad  with 
objects  of  various  compliances  during  passive  indentation.  Passive  touch  experiments  were 
conducted  by  using  the  tactile  stimulator  to  move  the  specimen  by  3mm  from  the  point  of 
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Figure  Nominal  contact,  area  versus  contact  force  for  specimens  of  various  softnesses 

for  sulrject  .JK. 


barely  touching  to  indent  the  stationary  finger  at  a  constant  speed  of  2.4mm/sec.  The  overall 
contact  areas  were  obtained  by  using  low  magnification  to  capture  the  entire  contact  region 
when  the  forces  were  less  than  IN.  At  higher  contact  forces,  the  low  magnification  images 
obtained  to  capture  the  entire  contact  region  were  of  poor  quality  due  to  the  deformation  of 
the  specimens.  Therefore,  the  percentage  contact  areas  were  obtained  separately  using  high 
magnification.  Since  the  real-time  imaging  setup  does  not  have  a  large  depth  of  focus  at  high 
magnification,  the  focus  was  adjusted  to  about  half  of  the  range  of  specimen  motion  to  obtain 
acceptable  images  during  indentation. 

The  overall  contact  boundaries  were  extracted  manually  since  the  automatic  method  did 
not  give  satisfactory  results  for  all  the  images  due  to  the  presence  of  noise  and  specimen 
deformation.  The  area  calculations  were  performed  on  the  manually  extracted  borders  which 
were  obtained  by  using  MATLAB  software  GINPUT  function.  The  overall  area  versus  mean 
contact  force  curves  for  all  the  specimens  were  about  the  same  when  the  contact  force  was 
less  than  about  IN.  Figure  4-15  shows  the  results  of  nominal  contact  area  versus  the  contact 
forces  from  three  trials,  plotted  with  the  same  symbol,  of  passive  touch  experiments  with 
various  specimens,  plotted  with  different  symbols,  for  subject  JK. 

To  have  higher  resolution  in  the  data  analysis,  the  percentage  contact  areas  for  soft  spe¬ 
cimens  were  obtained  with  higher  magnification.  By  using  homomorphic  processing,  the 
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Figure  4-16:  Percentage  contact  data  from  three  trials  for  subject  CH  in  contact  with 
soft  specimen  CA4(0.304mm/N). 
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Figure  4-17:  Percentage  contact  data  from  three  trials  for  subject  CH  in  contact  with  the 
hard  specimen  AB4(0.086mm/N). 


portions  that  were  in  contact  were  separated  from  the  portions  that  were  not  in  contact. 
Figure  4-16  shows  the  results  of  three  trials,  plotted  using  three  different  symbols,  from 
contact  experiments  using  a  compliant  object  (CA4)  which  has  an  objective  compliance  of 
0.304mm/N.  The  mean  contact  force  during  ea.ch  image  frame  increased  as  the  specimen 
was  indented  into  the  finger,  by  the  stimulator.  The  corresponding  percentage  contact  area 
grew  with  increased  depth  of  indentation.  Since  the  data  was  processed  using  homomorphic 
processing,  the  percentage  results  which  are  less  than  15%  or  more  than  85%  are  considered 
unreliable  because  the  processed  image  histogram  was  approximately  unimodal  in  the  corres¬ 
ponding  frames  and  an  appropriate  threshold  value  could  not  be  found  easily.  Also,  even  if 
the  resulting  histogram  is  bimodal,  a  threshold  value  which  differs  slightly  would  give  slightly 
different  results.  Therefore,  errors  of  a  few  percent  are  expected  and  should  be  kept  in  mind 
when  interpreting  the  data.  The  data  from  the  three  trials,  shown  in  different  symbols,  are 
very  consistent  as  indicated  by  the  small  variations  in  the  results.  The  percentage  contact 
rose  quickly  to  40%  with  a  contact  force  of  only  0.25N.  As  the  contact  force  increased  to  3N, 
the  percentage  contact  rose  more  graduall\'  to  about  60%. 
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SubjectiCH  Specimens:  hard(x),  medium(4),soft(o) 
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Figure  4-18:  Percentage  contact  results  from  various  soft  specimens  used  for  subject  CH 


Subject:CT  Specimens:  hard{x),  medium(+),soft(o) 


Figure  4-19:  Percentage  contact  results  from  various  specimens  used  for  subject  CT. 


Figure  4-20:  Percentage  contact  results  from  various  specimens  used  for  subject  JK. 


Figure  4-21:  Comparison  of  percentage  contact  results  from  various  subjects  for  the  hard 
specimen. 
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Specimen:mediuni  Subject:  CH(o),  CT(x},  JK(+) 


Contact  Force  (N) 

Figure  4-22;  Comparison  of  percentage  contact  results  from  various  subjects  for  the  medium 
specimen. 
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Figure  4-23:  Comparison  of  percentage  contact  results  from  various  subjects  for  the  soft 
specimen. 
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The  results  of  contact  experiments  with  harder  specimen  AB4  which  has  an  objective 
compliance  of  only  0.086mm/N  are  shown  in  Figure  4-17.  For  the  three  trials  conducted, 
there  is  slightly  more  variability  among  the  results  compared  to  those  for  the  soft  specimen. 
The  resulting  percentage  contact  was  only  about  40%  at  3N  force,  which  is  significantly  less 
than  that  for  the  soft  specimen.  The  results  for  the  three  different  soft  specimens  used  are 
comi^ared  in  Figure  4-18  for  subject  CH.  The  results  from  subject  CT  and  JK  for  various 
s[3ecimens  are  shown  in  Figures  4-19  and  4-20,  respectively.  The  percentage  contact  area  is 
generally  lower  for  the  specimens  with  lower  objective  compliance.  Comparison  of  the  results 
for  the  three  subjects  in  contact  with  various  specimens  used  is  shown  in  Figures  4-21  to 
4-23.  .Although  there  is  some  variability  among  the  results  for  different  subjects,  the  effect  of 
subject  was  not  as  ol^vious  as  the  effect  of  specimens. 

4.5.3  Description  of  slip 

To  observe  contact  phenomena  accompanying  slip,  the  videomicroscopy  system  was  setup  to 
record  the  contact  region  as  the  subject  actively  stroked  the  fixed  Plexiglas  specimen  across 
the  field  of  view  of  the  camera.  By  viewing  the  image  sequence  composed  of  frames  taken  50 
msec  apart,  one  realizes  that  as  the  shear  force  applied  on  the  finger  increases,  slip  between 
the  fingerpad  and  the  plate  starts  from  the  border  of  the  contact  region.  The  region  which 
remains  stationary  in  the  center  shrinks  as  time  progresses.  Eventually,  the  whole  contact 
region  slides  as  a  whole  across  the  surface  of  the  plate  and  the  image  of  the  contact  region 
remains  unchanged  as  in  rigid  body  translation.  The  figures  described  below  illustrate  this 
phenomenon  in  greater  detail. 

The  relative  motion  of  the  finger  ridges  can  be  observed  by  comparing  the  scan-lines  of 
two  adjacent  frames.  In  Figure  4-24,  images  A  and  B  are  50  msec  apart  in  time.  Although 
they  look  very  similar  to  the  eye,  there  are  slight  differences  in  the  locations  of  the  finger 
ridges,  especially  in  the  left  and  right  portions  of  the  image.  The  gray  scale  variations  on 
line  240,  the  middle  horizontal  line  of  the  640  by  480  size  images,  is  plotted  in  the  lower  left 
graph.  The  gray-scale  variation  from  image  A  is  plotted  as  a  solid  line  and  that  from  image  B 
is  plotted  as  a  dotted  line.  The  dotted  line  mostly  coincides  with  the  solid  line  in  the  middle 
portion  of  the  scan  line  and  leads  the  solid  line  on  the  two  sides.  The  differences  of  the  two 
scan-lines  are  shown  in  the  lower  right  graph. 

The  movement  of  the  finger  ridges  in  consecutive  frames  is  depicted  in  Figure  4-25.  The 
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Figure  4-24:  Typical  images  and  scan-lines  from  contact  between  a  fingerpad  and  a  Plexiglas 
plate  when  slip  started  to  occur  on  the  boundary. 


Image  1,  Line  240 


Figure  4-25:  Comparison  of  gray  scale  values  on  a  scan-line  in  a  sequence  of  images  at  the 
incipience  of  slip. 


top  three  graphs  consist  of  gray  scale  variation  on  a  scan-line  in  the  center  of  three  consecutive 
images  while  the  bottom  graph  is  a  superposition  of  the  three.  Grid  lines  are  provided  for 
easy  comparison  of  finger  ridges.  By  comparing  the  finger  ridges,  it  is  concluded  that  the 
middle  region  has  not  moved  yet  in  all  the  three  frames  which  occur  over  150  msec  in  time, 
whereas  the  two  ends  have  already  moved  an  observable  amount.  The  distances  between 
ridges  on  the  left  side  of  the  graph  are  stretched  whereas  the  middle  region  is  still  stationary. 
In  contrast,  the  distances  between  the  ridges  on  the  right  are  compressed. 

Although  this  phenomenon  of  slip  starting  from  the  boundary  and  propagating  inwards 
can  be  seen  vividly  by  playing  the  recorded  image  sequences  in  real-time,  it  is  difficult  to 
present  such  results  in  stationary  pictures  on  a  single  page.  By  superimposing  the  finger  ridge 
outlines  of  the  image  sequences,  such  effects  can  be  seen  more  easily.  The  left  hand  graphs 
of  Figure  4-26  are  three  images  from  the  slip  sequence.  The  outlines  of  the  finger  ridges  are 
obtained  by  morphological  operations,  as  explained  in  the  image  processing  section.  In  the 
upper  right  graph,  the  outlines  of  the  first  image  is  shown.  In  the  middle,  the  outlines  of  both 
first  and  second  images  are  shown.  This  is  created  by  performing  a  logical  OR  operation 
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from  the  two  outline  images.  In  the  bottom,  the  outlines  generated  from  the  three  frames  are 
shown.  By  comparing  the  three  outline  graphs,  one  sees  immediate^  that  the  regions  in  the 
middle  have  not  moved  while  the  borders  have  moved  an  appreciable  amount. 

4.6  Discussion 

4.6.1  Contact  phenomena 
The  development  of  contact 

Based  on  the  images  and  force  data  obtained  from  the  low  magnification  experiments  using  a 
rigid  Plexiglas  plate,  our  understanding  of  the  development  of  contact  has  been  enhanced.  By 
combining  the  information  available  in  variables  such  as  nominal  (overall)  contact,  areas,  ac¬ 
tual  contact  areas,  and  contact  forces,  various  pieces  of  information  generated  during  contact 
can  be  calculated  (see  Figures  4-13  and  4-14).  Surprisingly,  the  actual  contact  area  increased 
slowly  as  compared  to  the  nominal  area  during  the  indentation,  which  indicates  presence  of 
viscoelasticity  in  the  fingerpad.  At  the  same  time,  the  actual  mean  contact  pressure  did  not 
vary  much  as  compared  to  the  rate  of  change  in  nominal  mean  contact  pressure.  On  the  other 
hand,  the  percentage  contact  increased  during  the  indentation  phase  and  remained  roughly 
constant  during  the  retraction  phase. 

The  results  from  visualization  of  contact  region  showed  a  considerable  difference  between 
loading  and  unloading  in  terms  of  contact  pressure,  contact  area  and  the  percentage  contact 
inside  the  nominal  contact  region.  Soft  tissues  which  make  up  the  fingerpad  can  be  modeled 
according  to  Fung’s  [16]  quasi-linear  viscoelastic  model,  which  was  proposed  to  explain  the 
mechanical  properties  of  biological  tissues  with  a  response  consisting  of  an  elastic  response 
and  a  reduced  relaxation  function.  The  increased  actual  area  of  contact  during  the  loading 
phase  can  also  be  thought  of  as  more  and  more  springs  coming  into  contact  from  each  of 
the  finger  ridge  (small  contact  islands  growing  bigger  and  getting  connected  and  the  ridges 
growing  fatter  due  to  increases  in  the  contact  force  applied).  During  the  unloaxiing  phase, 
however,  the  same  springs  do  not  retract  due  to  viscoelastic  effects  (as  shown  by  the  slow 
decrease  of  actual  contact  area  in  Figure  4-14).  Instead,  it  is  likely  that  the  stiffness  of 
each  spring  reduces  as  the  contact  force  is  reduced.  Eventually,  the  ridges  get  thinner  and 
disconnect  and  finally  disappear. 
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Image  1 


Outline  1 


Figure  4-26:  Visualizing  the  slip  sequence  in  action  by  combining  images  processed  by  the 
outlining  procedure. 
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The  hypothesis  of  pressure  distribution 

Although  the  contact  information  discussed  used  mean  pressure  inside  the  contact  region,  the 
actual  distribution  is  quite  likely  to  be  non-uniform.  Without  high  resolution  pressure  sensor 
arrays,  the  pressure  distribution  cannot  be  measured  directly.  The  pressure  distribution 
that  one  would  expect  from  contact  mechanics  [23]  is  parabolic  in  nature,  assuming  that  the 
materials  in  contact  are  linear. 

If  we  assume  that  the  material  properties  of  the  individual  finger  ridges  are  similar,  the 
static  friction  coefficient  between  the  finger  ridges  and  the  Plexiglas  would  also  be  similar. 
During  active  slip,  we  observed  that  the  boundaries  of  the  contact  region  slipped  first,  whereas 
the  center  of  the  contact  region  remained  stationary.  One  reasonable  hypothesis  for  the  border 
to  slip  first  is  that  the  pressure  distribution  on  the  contact  region  is  in  fact  not  uniform. 
The  pressure  in  the  middle  is  probably  higher  than  that  in  the  surround,  as  expected  from 
mechanics.  Therefore,  the  shear  force  overcomes  the  static  friction  at  the  borders  first  and 
the  middle  slipped  only  when  the  shear  force  fully  overcame  the  frictional  forces.  In  addition, 
during  such  a  process,  the  finger  ridge  distances  in  the  front  portion  of  the  motion  get 
stretched  because  the  middle  region  has  not  moved  yet  and  the  ridges  on  the  back  portion  of 
the  fingerpad  get  compressed.  From  a  neurophysilogical  viewpoint,  the  skin  stretching  and 
compression  could  be  sensed  by  the  mechanoreptors  in  order  to  detect  the  incipience  of  slip 
(Srinivasan  et  al  [43]). 

4.6.2  Contact  information  available  for  softness  discrimination 

During  softness  discrimination  experiments  using  normal  fingers  to  perform  discrimination, 
the  subjects  had  tactile  information  from  the  contact  region  in  addition  to  kinesthetic  in¬ 
formation.  From  the  visualization  of  fingerpad  contact  with  soft  specimens,  we  know  that  the 
overall  contact  areas  were  about  the  same  for  the  hard  (0.086mm/N),  medium  (0.188mm/N), 
and  soft  (0.304mm/N)  specimens  when  the  contact  forces  were  up  to  IN  (Figure  4-15).  The 
percentage  contact  areas  were,  however,  significantly  different  (Figures  4-18,  4-19,  and  4-20) 
for  specimens  of  various  softness.  Thus,  if  we  assume  that  the  overall  contact  areas  were 
about  the  same  for  the  different  specimens  even  for  higher  contact  forces,  the  percentage 
contact  areas  and/or  the  pi'essure  distributions  could  be  the  cues  for  subjects  to  discriminate 
object  softness. 


Since  the  subjects  reported  that  the  discriminations  were  based  on  the  indentation  portion 
of  the  contact,  the  retraction  portion  of  the  fingerpad  contact  was  not  important  for  this 
task.  When  the  subjects  wore  thimbles,  the  contact  interface  conditions  were  different  from 
the  normal  finger  conditions.  The  tactile  information  which  allowed  the  subjects  to  have  fine 
softness  discriminability  was  eliminated  and  the  subjects’  performance  degraded  significantly. 
When  the  subjects  had  thin  (0.06mm)  finger  cots  on,  the  performance  was  not  statistically 
different  from  the  normal  finger,  which  suggests  that  the  finger  ridge  level  details  are  not 
important  for  discriminating  the  specimens  used. 

4.6.3  The  role  of  contact  mechanics  in  haptic  interaction 

During  haptic  interaction  with  objects,  be  it  exploration  or  manipulation,  tactile  sensory 
information  originates  directly  from  the  contact  region.  The  development  of  contact  region 
depends  not  only  on  the  gross  properties  of  objects  and  fingers,  but  also  on  the  detailed 
mechanics  of  contact,  the  underlying  pressure  distributions,  as  well  as  the  history  of  the 
contact.  For  a.  seemingly  simple  task  such  as  handling  an  egg  during  its  transportation  from 
the  refrigerator  to  the  kitchen  counter,  our  fingers  need  to  exert  the  right  amount  of  force  to 
not  break  or  drop  the  egg. 

From  the  visualization  of  the  slip  phenomenon,  we  see  that  slipping  of  the  finger  ridges 
near  the  boundaries  of  contact  occured  first,  which  caused  the  finger  ridges  to  stretch  in  front 
and  compress  near  the  back.  These  relative  movements  of  the  finger  ridges  will  most  likely 
cause  the  mechanoreceptors  underneath  the  skin  to  respond  and,  through  the  central  nervous 
system,  notify  the  brain  about  the  incipience  of  slip,  so  that  the  brain  can  activate  the  motor 
subsystem  to  take  appropriate  actions  to  avoid  objects  from  slipping  through  our  fingers. 

During  indentation,  the  fast  growth  of  the  overall  (nominal)  contact  region  allows  the 
contact  forces  to  be  distributed  across  a  large  region  in  a  short  period  of  time  to  provide  stable 
contact.  The  actual  area  of  contact  developed  more  slowly  in  the  meantime,  as  described 
in  the  sections  on  contact  areas.  The  retraction  or  unloading  portion  of  the  contact  showed 
mechanisms  that  provide  as  large  an  area  of  contact  as  possible  by  reducing  the  actual 
pressure,  but  maintaining  the  overall  contact  area  for  a  prolonged  period  of  time  to  provide 
stable  contact. 

With  these  additional  insights  provided  by  this  study  on  contact  visualization,  it  seems 
clear  that  the  same  kind  of  contact  process  which  depends  a  lot  on  the  mechanics  of  contact 
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has  to  happen  in  order  to  achieve  successful  haptic  interaction  with  more  complex  objects 
as  well.  It  may  also  be  helpful  to  design  robotic  fingers  to  have  similar  relationship  among 
contact  forces,  nominal  areas,  and  actual  areas. 
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Chapter  5 


Conclusions 

5.1  Main  results 

5.1.1  Softness  discrimination 

To  measure  the  abilities  of  humans  in  discriminating  softness  with  their  fingerpads  bj’  active 
touch  (with  eyes  closed),  a  specimen  presenter  system  was  built  to  randomly  present  the 
softness  specimens  for  the  discrimination  task.  Two  experimental  paradigms  Involving  one 
and  two  index  fingers  were  used;  three  finger-contact  interface  conditions  were  tested  (normal, 
finger  cot,  and  rigid  thimble).  The  results  from  the  three  different  finger  conditions  show  that 
the  fine  detailed  finger  ridge  level  tactile  information  is  not  necessary  for  the  task  because 
discrimination  with  the  finger  cot  is  as  good  as  with  the  normal  finger  (see  Tables  2.3  and  2.4) . 
But  when  thimbles  were  present,  the  dramatic  degradation  in  discriminability  demonstrated 
the  absolute  necessity  of  tactile  information  to  achieve  normal  resolution. 

In  order  to  look  into  the  strategies  and  underlying  mechanisms  used  during  the  softness 
discrimination  process,  an  experimental  paradigm  involving  the  two  index  fingers  from  both 
hands  was  used.  The  performance  under  both  normal  and  finger  cot  conditions  improved, 
which  could  be  attributed  to  the  fact  that  in  this  paradigm,  the  subjects  need  only  to  rely 
on  immediately  available  sensory  information  and  not  on  memory.  When  the  peak  forces 
applied  with  normal  finger  on  the  two  specimens  by  each  subject  are  compared,  three  different 
strategies  stand  out:  (1)  to  achieve  the  same  amount  of  deformation  of  the  two  specimens 
by  exerting  higher  forces  on  the  harder  specimen,  (2)  to  compare  the  contact  information 
obtained  by  applying  the  same  amount  of  pressure  on  both  specimens  (thus  higher  peak 


forces  on  the  softer  specimen),  and  (3)  to  apply  the  same  amount  of  peak  forces  on  both 
specimens  and  compare  the  deformations. 

Although  one  of  the  subjects  consistently  applied  higher  forces  with  the  right  hand,  the 
subject’s  performance  did  not  seem  to  be  affected,  implying  that  the  brain  can  compensate 
for  such  differences. 

5.1.2  Force  control 

In  the  force  control  experiments,  subjects  were  asked  to  achieve  and  maintain  a  constant 
contact  force  by  pressing  their  fingerpads  on  specimens  with  the  following  experimental  para¬ 
meters:  object  softness,  contact  interface  condition  (normal,  finger  cot,  tliimble),  target  force 
magnitude  (2N,  4N,  8N),  visual  feedback  availability,  and  the  hand  used.  Statistical  tests 
on  the  results  indicated  that  the  errors  from  tracking  with  visual  feedback  was  significantly 
lower  than  when  the  subjects  had  to  maintain  the  force  without  visual  feedback.  The  abil¬ 
ities  of  the  subjects  to  control  forces  on  specimens  of  various  softness  under  the  three  finger 
contact  interface  conditions  were  not  significantly  different.  The  errors  were  higher  when 
controlling  a  higher  target  force.  In  addition,  significant  differences  were  found  between  the 
force  control  abilities  of  the  two  hands  for  the  subject  who  showed  handedness  in  the  softness 
discrimination  experiments. 

5.1.3  Contact  visualization 

A  real-time  imaging  setup  which  integrated  a.  videomicroscopy  system  with  a  tactile  stimulator 
system  by  means  of  a  custom  made  communication  interface  was  developed.  This  system 
allowed  both  the  contact  images  and  the  contact  forces  to  be  digitized  in  real-time.  This 
setup  was  used  for  visualizations  of  the  overall  and  actual  contact  areas  as  well  as  for  detailed 
observation  of  the  phenomenon  of  slip. 

Various  image  processing  techniques  were  developed  and  applied  in  order  to  obtain  con¬ 
tact  information  from  the  digitized  images.  The  images  of  the  contact  regions  were  improved 
by  first  applying  homomorphic  processing  for  contrast  enhancement  and  then  performing 
a  global  thresholding  or  scan-line  processing.  Homomorphic  processing  is  better  suited  for 
images  with  larger  contact  regions,  since  it  is  likely  to  produce  a  a  bimodal  histogram  which 
benefits  the  thresholding  step  for  converting  the  original  image  into  a  binary  image  consisting 
of  contact  and  non-contact  regions.  The  overall  contact  area  in  each  image  was  estimated 
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by  using  a  modified  wedge-ring  approach  to  determine  the  approximate  boundaries  of  the 
contact  images  obtained  with  low  magnification. 

Based  on  the  images  and  force  data  obtained  from  the  low  magnification  experiments  us¬ 
ing  rigid  Plexiglas,  and  examining  the  mean  pressure  on  the  contact  regions,  various  sources 
of  tactile  information  such  as  overall  contact  areas,  actual  contact  areas,  percentage  con¬ 
tact,  nominal  mean  pressure,  and  actual  mean  pressure  were  calculated.  Surprisingly,  the 
actual  contact  area  increased  much  more  slowly  than  the  nominal  area  during  the  indenta¬ 
tion.  But  the  actual  contact  pressure  did  not  vaiy  as  much  as  the  nominal  contact  pressure 
during  indentation.  On  the  other  hand,  the  percentage  contact  increased  steadily  during  the 
indentation  and  remained  roughly  constant  during  retraction. 

The  results  of  visualization  of  the  fingerpad  in  contact  with  soft  specimens  showed  that  the 
overall  contact  areas  were  about  the  same  for  various  specimens  for  contact  forces  up  to  IN. 
However,  the  i)ercentage  contact  area  inside  the  contact  region  obtained  from  high  magnifica¬ 
tion  images  showed  significant  differences  for  the  hard  (0.086mm/N),  medium  (0.188mm/N), 
and  soft  (0.304mm/N)  specimens. 

From  the  visualization  of  the  slip  between  the  fingerpad  and  a  rigid  Plexiglas  plate,  it 
was  shown  that  slipping  first  occured  only  near  the  boundaries  of  contact,  which  caused  the 
finger  ridges  to  stretch  in  front  and  compress  near  the  back.  These  relative  movements  of 
the  finger  ridges  most  likely  cause  the  mechanoreceptors  underneath  the  skin  to  respond  and, 
through  the  central  nervous  system,  notify  the  brain  about  the  incipience  of  slip.  The  brain 
can  then  activate  the  motor  subsystem  to  take  appropriate  action  to  avoid  slipping  of  objects 
through  our  fingers. 

5.2  Implications  and  applications 

5.2.1  Human  haptic  interaction  with  soft  objects 

Although  the  process  of  the  haptic  interaction  with  objects,  which  generally  involves  mul¬ 
timodal  perception,  and  intersensory  integration  during  exploration  or  manipulation,  is  com¬ 
plicated,  our  understanding  of  the  process  has  been  improved  by  this  investigation  of  the 
phenomenon  from  three  different  perspectives:  discriminability,  force  control,  and  contact 
visualization. 

The  abilities,  at  the  behavioral  level,  to  distinguish  fine  differences  in  object  softness 
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without  useful  visual  and  auditory  information  are  likely  to  depend  on  both  tactual  percep¬ 
tion  and  motor  control.  The  subject  with  poorer  performance  in  controlling  forces  without 
visual  feedback  of  force,  also  did  not  do  as  well  in  discrimination  when  compared  to  the 
other  subjects.  Memory  does  play  a  role  in  discriminability  as  shown  from  the  significant 
improvement  in  the  results  for  all  subjects  when  using  both  index  fingers  to  discriminate  the 
two  specimens.  Although  subjects  may  exhibit  handedness  as  shown  from  the  differences  in 
force  control  ability  of  the  two  fingers  and  the  large  difference  of  forces  used  during  discrim¬ 
ination,  the  ability  to  discriminate  was  not  impaired,  which  indicates  that  the  brain  was  able 
to  accommodate  for  handedness  during  the  cognition  process. 

Since  tactile  information  is  essentia]  to  achieve  normal  softness  resolution,  the  mechanisms 
for  discrimination  must  rely  on  the  spatio-temporal  profiles  of  various  contact  variables. 
However,  as  seen  in  the  visualization  experiments,  our  finger  has  the  property  of  developing 
the  nominal  contact  area  quickly  even  with  a  small  force.  The  overall  contact  areas  were 
about  the  same  for  specimens  of  different  softness  for  forces  up  to  IN.  But  the  actual  contact 
areas  were  very  different,  based  on  high  resolution  images  obtained  with  high  magnification. 
.\ssuming  that  the  overall  contact  areas  were  also  about  the  same  for  specimens  of  different 
softnesses  at  higher  contact  forces,  the  information  used  during  discrimination  is  probably 
the  spatio-temporal  distribution  of  the  pressure  over  the  nominal  contact  region.  According, 
to  Srinivasan  and  LaMotte  [42] [40] [41],  this  information  can  be  signaled  by  mechanoreceptors 
of  the  type  SAI  (Merkel  cells)  and  RAI  (Meissner  corpuscles). 

5.2.2  The  pressure  distribution  hypothesis 

If  the  material  properties  of  the  individual  finger  ridges  are  assumed  to  be  similar,  the  static 
friction  coefficient  between  individual  finger  ridges  and  the  Plexiglas  would  also  be  similar. 
In  the  visualization  of  the  slip  phenomenon,  we  observed  that  the  boundaries  of  the  contact 
region  moved  first  whereas  the  center  of  the  contact  slipped  last.  The  slip  front  moved 
gradually  inwards  during  the  process  and  then  disappeared,  at  which  time  the  finger  ridges 
seemed  to  translate  as  a  rigid  bod3C  One  reasonable  hypothesis  for  the  boundaries  to  slip  first 
is  that  the  pressure  distribution  on  the  contact  region  is  in  fact  not  uniform.  If  the  contact 
pressure  in  the  middle  is  higher  and  decreased  either  linearly  or  non-linearly  until  it  reached 
the  edge,  the  observed  slip  phenomenon  can  be  explained  as  due  to  Coulomb  friction  laws. 


5.2.3  Slip  detection 


During  the  slip  process,  the  finger  ridges  at  the  boundaries  overcome  the  static  Coulomb 
friction  first  and  moved  relative  to  the  center  of  the  contact  region.  The  middle  portion  onh' 
slipped  when  the  total  shear  force  overcame  the  total  frictional  force.  The  inter-ridge  distance 
in  the  front  portion  of  the  contact  region  got  stretched  because  the  middle  region  had  not 
moved  yet.  In  contrast,  the  ridges  on  the  back  portion  of  the  fingerpad  got  compre.ssed.  From 
a  neurophysiological  viewpoint,  the  skin  stretching  and  compressing  could  be  the  mechanical 
stimulus  sensed  by  the  mechanoreceptors  and  the  corresponding  neural  impulses  transmitted 
to  the  brain  could  be  used  for  slip  detection  and  prevention. 

5.2.4  Dexterous  robot  finger  design 

If  we  imagine  the  human  as  the  ultimate  robot,  then  building  a  dexterous  robot  should  borrow 
as  much  of  the  human  system  design  or  functionality  as  possible.  During  object  manipulation, 
the  control  of  contact  forces  is  a  major  concern.  An  ideal  dexterous  robot  finger  should  be 
able  to  maintain  stable  contact  with  objects  having  a  wide  range  of  properties.  Although  an 
intelligent  control  system  which  drives  such  a  robot  can  use  force  sensors  available  in  the 
various  joints  to  sense  the  environmental  properties  and  to  perform  dexterous  manipulations, 
it  would  be  beneficial  to  construct  robot  fingers  that  can  exhibit  some  of  the  desired  contact 
phenomena  revealed  in  this  study.  The  development  of  a  robot  fingerpad  with  a  fast  growth 
of  the  overall  (nominal)  contact  region  as  contact  forces  are  increased  allows  the  contact 
forces  to  be  distributed  across  a  large  region  in  a  short  period  of  time  to  provide  stable 
contact.  The  actual  area  of  contact  needs  to  develop  gradually,  in  the  meantime,  so  as  to 
provide  a  gradual  increase  in  contact  pressure.  During  the  retraction  or  unloading  portion 
of  the  contact,  if  the  mechanisms  were  to  maintain  a  large  area  of  contact  even  when  force 
dropped,  the  actual  pressure  would  reduce  and  the  contact  would  be  more  stable.  The  other 
benefit  of  providing  a  large  contact  surface  as  opposed  to  a  point  contact  (as  in  the  case  of 
a  rigid  robot  finger)  is  to  simultaneously  avoid  crushing  of  objects  with  the  same  amount  of 
force  but  lower  pressure  and  prevent  slip  with  a  lower  force  to  conserve  energy. 
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5.2.5  Biomechanical  visualization 


The  videomicroscopy  and  tactile  stimulator  systems  developed  during  this  study  are  powerful 
tools  to  investigate  the  underlying  mechanics  of  contact.  The  videomicroscopy  system  was 
used  by  Dandekar  [10],  a  recent  Ph.D.  from  Touchlal),  MIT,  to  successfully  obtain  images  of 
the  fingertip  required  for  3D  reconstruction  and  the  surface  deformation  experiments  which 
were  useful  in  verifying  finite  element  models  of  the  fingertip.  The  real-time  imaging  setup 
has  proven  to  be  very  useful  in  revealing  the  underlying  mechanics  of  contact.  Several  pieces 
of  contact  information,  such  as  the  overall  contact  area,  actual  contact  area,  nominal  contact 
pressure,  and  actual  pressure,  were  measured  l)a,sed  on  the  image  and  force  data.  The 
observations  of  slip  led  to  the  hypothesis  of  non-uniform  pressure  distribution  within  the 
contact  region  which  is  yet  to  be  proved  in  the  future,  possibly  with  high  resolution  tactile 
sensor  arrays.  Other  possible  experiments  include  observations  of  the  finger  ridge  deformation 
as  the  fingerpad  performs  scanning  motion  to  detect  surface  irregularities. 

5.2.6  Automated  fingerprint  identification  system 

Automated  fingerprint  identification  systems  could  benefit  from  the  results  of  this  study.  In 
terms  of  real-time  fingerprint  image  acquisition,  the  developed  videomicroscopy  system  is  cap¬ 
able  of  not  only  static  images  but  also  dynamic  images.  The  slip  phenomenon,  prerecorded 
for  each  individual  using  the  system,  can  be  used  as  test.  Such  operation  reduces  the  risk  of 
beating  the  system  with  artificial  impostor  fingers,  which  would  not  have  the  same  mechanical 
properties  as  live,  real,  fingers  of  a  particular  individual.  The  scan-line  processing  techniques 
can  be  implemented  in  real-time  to  convert  the  fingerprint  images  into  binary  format,  for 
identification  or  storage  purposes.  As  for  image  storage,  a  typical  compression  ratio  of  30 
was  obtained  by  applying  LZW  compression  algorithms,  widely  used  in  Unix  systems,  to 
the  binary  fingerpad  images  to  achieve  lossless  compression.  With  lossless  compression,  the 
minute  details  of  the  finger  ridges,  sweat  pores,  and  the  small  islands  between  ridges  can  be 
preserved.  These  techniques  can  be  used  to  replace  the  high  quality  lossy  compression  stand¬ 
ard,  Wavelet  Scalar  Quantization  (WSQ)  [3]  [4],  currently  adopted  by  the  Federal  Bureau  of 
Investigation  (FBI). 
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Appendix  A 


Softness  Discrimination:  Analysis  of 
Applied  Forces 


This  section  presents  the  graphs  similar  to  Figure  2-13  for  the  peak  forces  used  in  S-2AFC 
discrimination  tasks.  Peak  forces  on  soft  specimens  versus  peak  forces  on  hard  specimens  for 
subjects  CH,  CT,  and  .JK  under  normal,  finger  cot,  and  thimble  conditions  are  plotted.  The 
four  panels  are  for  the  four  specimen  pairs.  The  data  were  pooled  from  3  experimental  runs, 
64  trials  each,  conducted  for  each  of  the  experimental  conditions.  The  percentage  of  correct 
calls,  confusion  matrix  (labeled  CM),  and  force-specimen-correctness  matrix  (labeled  FsM) 
are  also  shown  in  each  panel. 
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Figure  A-1:  S-2AFC  Peak  force  plot  for  subject  CH  under  normal  condition. 


Data:  ctdr3cr3.fln,  Spe.  Diff.:  6.7% 


Gorrect(%):  65 
z40  GM;  73/23/44/52 
r  FsM:  59/39/66/28 


10  20  30  40 

Peak  Force  on  Hard  (N) 

Data:  cter3cr3.fln,  Spe.  Diff.:  16.2% 


Gorrect(%):  83 
GM;  80/l  6/1 7/79  • 
FsM:  64/17/95/16 


4 

.  CP  <9 

iP 

*b 

°  0 

X  0 

_Q_ 

10  20  30  40 

Peak  Force  on  Hard  (N) 


Data:  cter4cr3.fln,  Spe.  Diff.:  12.2% 
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Figure  A-2:  S-2AFC  Peak  force  plot  for  subject  CT  under  normal  condition. 
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Figure  A-3:  S-2AFC  Peak  force  plot  for  subject  .JK  under  normal  condition. 
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Figure  A-4:  S-2AFC  Peak  force  plot  for  subject  CH  under  finger  cot  condition. 


118 


Data:  ctdr4cr2.fan,  Spe.  Diff.:  8.7% 
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Figure  A-5:  S-2AFC  Peak  force  plot  for  subject  CT  under  finger  cot  condition. 
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Figure  A-6:  S-2AFC  Peak  force  plot  for  subject  JK  under  finger  cot  condition. 
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Figure  A-7:  S-2AFC  Peak  force  plot  for  subject  CH  under  thimble  condition. 
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Figure  A-8:  S-2AFC  Peak  force  plot  for  subject  CT  under  thimble  condition. 
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Data:  jkfr6cr6.frn,  Spe.  Diff.:  30.8% 
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Figure  A-9:  S-2AFC  Peak  force  plot  for  subject  JK  under  thimble  condition. 
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Appendix  B 


Softness  Discrimination;  JND  Data 


The  discrimination  results  from  an  experimental  run  was  represented  b,y  a  confusion  matrix. 
The  matrices  from  three  runs  of  64  trials  each  was  pooled  together  for  the  calculation  of  the 
sensitivity  index  d'  and  the  response  bias  /3  for  a  given  comparison  specimen.  Recall  that 
the  .JND  value  is  defined  as  the  difference  of  stimuli  which  corresponds  to  a  d'  value  of  1. 
The  .JND  value  for  a  particular  subject  under  a  given  finger  condition  was  then  obtained 
by  averaging  the  estimates  of  .JND  from  the  four  comparison  specimens.  The  data  points 
which  had  a  d'  value  less  than  0.5  was  not  included  in  the  mean  JND  calculations  and  their 
estimated  JND  values  are  enclosed  in  brackets. 
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Specimen 

DRl 

DR5 

ERl 

FRl 

Difference 

2.6% 

7.1% 

13.5% 

31.7% 

Confusion 

55  41 

74  22 

74  22 

92  4 

Matrix 

29  67 

20  76 

17  79 

1  95 

d' 

0.70 

1.55 

1.67 

4.04 

/? 

-0.17 

-0.04 

-0.09 

-0.29 

JND  (%) 

3.8 

4.6 

8.1 

7.8 

MeanJND  =  6.1% 

Table  B.l:  1I-2AFC  discrimination  results  for  subject  CH  under  normal  condition  (Standard: 
CRl  [0.304mm/N]) 


DR4 

ER4 

ER2 

Difference 

3.7% 

8.7% 

13.1% 

16.7% 

Confusion 

78  18 

69  27 

88  8 

Matrix 

13  83 

14  82 

7  89 

d' 

1.99 

1.63 

3.16 

2.84 

0 

-0.11 

-0.24 

0.05 

-0.04 

JND  (%) 

1.9 

5.3 

4.1 

5.9 

MeanJND  =  4.3% 

Table  B.2:  1I-2AFC  discrimination  results  for  subject  CH  under  finger  cot  condition  (Stand¬ 
ard:  CR2  [0.298mm/N]) 


Specimen 

ER4 

FR4 

HT2 

IT2 

Difference 

14.7% 

27.2% 

53.3% 

75.2% 

Confusion 

54  42 

54  42 

68  28 

76  20 

Matrix 

37  59 

27  69 

28  68 

18  78 

d’ 

0.45 

0.74 

1.10 

1.70 

/3 

-0.08 

-0.21 

-0.00 

-0.04 

JND  (%) 

[32.7] 

36.9 

48.7 

44.3 

MeanJND  =  43.3% 

Table  B.3:  1I-2AFC  discrimination  results  for  subject  CH  under  thimble  condition  (Standard: 
CR4  [0.293mm/N]) 
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Specimen 

DR2 

DR3 

ER3 

FR3 

Difference 

mUkWjM 

6.7% 

Confusion 

72  24 

72  24 

86  10 

96  0 

Matrix 

25  71 

22  74 

8  88 

3  93 

d' 

H^li 

1.42 

2.64 

4.43 

-0.03 

-0.06 

0.35 

JND  (%) 

2.3 

4.8 

6.1 

6.3 

MeanJND  =  4.9% 

Table  B.4;  S-2AFC  discrimination  results  for  subject  CH  under  normal  condition  (Standard: 
CR3  [0.300mm/N]) 


Specimen 

DR2 

DRl 

DR4 

ER4 

Difference 

3.7% 

4.9% 

8.7% 

13.1% 

Confusion 

76  20 

80  16 

85  11 

90  6 

Matrix 

21  75 

23  73 

12  84 

4  92 

d' 

1.59 

1.68 

2.35 

3.27 

13 

0.02 

0.13 

0.03 

-0.10 

JND  (%) 

2.4 

2.9 

3.7 

4.0 

MeanJND  =  3.2% 

Table  B.5:  S-2AFC  discrimination  results  for  subject  CH  under  finger  cot  condition  (Stand¬ 
ard;  CR2  [0.298mm/N]) 


Specimen 

FR6 

HT2 

IT2 

JT2 

Difference 

30.8% 

.  55.0% 

77.1% 

110.3% 

Confusion 

51  45 

65  31 

66  30 

82  14 

Matrix 

41  55 

30  66 

26  70 

8  88 

d' 

0.26 

0.95 

1.10 

2.44 

/3 

-0.05 

-0.02 

-0.06 

-0.16 

JND  (%) 

[117.7] 

58.1 

70.2 

45.3 

MeanJND  =  57.9% 

Table  B.6;  S-2AFC  discrimination  results  for  subject  CH  under  thimble  condition  (Standard; 
CR6  [0.290mm/N]) 
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Specimen 

DRl 

ERl 

FRl 

HT6 

Difference 

2.6% 

13.5% 

31.7% 

47.9% 

Confusion 

46  50 

55  41 

77  19 

92  4 

Matrix 

32  64 

28  68 

22  74 

4  92 

d' 

0.38 

0.73 

1.59 

3.64 

(i 

-0.24 

-0.18 

0.05 

0.00 

JND  (%) 

[7.0]  ^ 

18.5 

19.9 

13.8 

MeanJND  =  17.4% 

Table  B.7:  1I-2AFC  discrimination  results  for  subject  CT  under  normal  condition  (Standard: 
CRl  [0.304mm/N]) 


Specimen 

DR4 

ER4 

ER2 

FR2 

Difference 

8.7% 

13.1% 

16.7% 

28.4% 

Confusion 

60  36 

67  29 

70  26 

72  24 

Matrix 

35  61 

22  74 

29  67 

17  79 

d' 

0.66 

1.26 

1.13 

1.60 

P 

-0.01 

-0.11 

0.05 

-0.13 

JND  (%) 

13.1 

10.4 

14.8 

17.7 

MeanJN  D  =  T 

1.0% 

Table  B.8:  1I-2AFC  discrimination  results  for  subject  CT  under  finger  cot  condition  (Stand¬ 
ard:  CR2  [0.298mm/N]) 


Specimen 

FR4 

HT2 

IT2 

JT2 

Difference 

27.2% 

53.3% 

75.2% 

108.0% 

Confusion 

58  38 

52  44 

63  33 

72  24 

Matrix 

48  48 

36  60 

22  74 

17  79 

d' 

0.26 

0.42 

1.14 

1.45 

P 

0.13 

-0.11 

-0.17 

-0.05 

JND  (%) 

[102.99] 

[126.06] 

65.8 

74.5 

MeanJND  =  70.2% 

Table  B.9:  1I-2AFC  discrimination  results  for  subject  CH  under  thimble  condition  (Standard: 
CR4  [0.293mm/N]) 
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Specimen 

DR3 

ER4 

ER3 

FR3 

Difference 

6.7% 

12.2% 

16.2% 

28.1% 

Confusion 

73  23 

76  20 

80  16 

86  10 

Matrix 

44  .52 

14  82 

17  79 

5  91 

d' 

0.81 

1.87 

1.89 

2.88 

P 

0..30 

-0.12 

0.02 

-0.18 

JND  (%) 

8.3 

6.6 

8.6 

9.7 

MeanJND  =  8.3% 

Table  B.IO:  S-‘2AFC  discrimination  results  for  subject  CT  under  normal  condition  (Standard; 
CR3  [0.300mm/N]) 


Specimen 

DR4 

ER4 

ER2 

FR2 

Difference 

8.7% 

13.1% 

16.7% 

28.4% 

Confusion 

69  27 

72  24 

80  16 

85  11 

Matrix 

42  54 

17  79 

15  81 

16  80 

d' 

0.74 

1.60 

1.98 

2.17 

P 

0.21 

-0.13 

0.02 

0.12 

-  .JND  (%) 

11.8 

8.2 

8.5 

13.1 

MeanJND  —  10.4% 

Table  B.ll:  S-2AFC  discrimination  results  for  subject  CT  under  finger  cot  condition  (Stand¬ 
ard;  CR2  [0.298mm/N]) 


Specimen 

FR6 

HT2 

IT2 

JT2 

Difference 

30.8% 

110.3% 

Confusion 

53  43 

60  36 

74  22 

Matrix 

54  42 

27  69 

29  67 

d' 

-0.03 

0.89 

0.90 

1.26 

P 

0.14 

0.10 

-0.13 

0.11 

JND  (%) 

[*] 

61.5 

86.0 

87.6 

MeanJND  =  78.4% 

Table  B.12;  S-2AFC  discrimination  results  for  subject  CT  under  thimble  condition  (Standard; 
CR6  [0.290mm/N]) 
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Specimen 

DRl 

DR.5 

ERl 

FRl 

Difference 

2.rj% 

7.1% 

13.5% 

31.7% 

Confusion 

Matrix 

69  27 

29  67 

64  32 

28  68 

77  19 

9  87 

88  8 
10  86 

d' 

1.10 

0.98 

2.17 

2.64 

ft 

0.03 

-0.06 

-0.24 

0.06 

JND  (%) 

2.4 

7.3 

6.2 

12.0 

MeunJND  =  7.0% 

Table  B.13:  1I-2AFC  discrimination  results  for  subject  JK  under  normal  condition  (Standard: 
CRl  [0.304mm/N]) 


Specimen 

DR2 

DR4 

ER4 

ER2 

Difference 

3.7% 

8.7% 

13.1% 

16.7%. 

Confusion 

61  35 

68  28 

86  10 

86  10 

Matrix 

20  76 

19  77 

3  93 

9  87 

d' 

1.16 

1.40 

3.12 

2..58 

ft 

-0.23 

-0.15 

-0.30 

-0.03 

JND  (%) 

3.2 

6.2 

4.2 

6.5 

MeanJND  =  5.0% 

Table  B.14:  1I-2AFC  discrimination  results  for  subject  JK  under  finger  cot  condition  (Stand¬ 
ard:  CR2  [0.298mm/N]) 


Specimen 

HT2 

IT2 

JT2 

Difference 

27.2% 

Confusion 

42  54 

57  39 

66  30 

Matrix 

23  73 

19  77 

15  81 

5  91 

d' 

0.55 

1.09 

1.50 

2.20 

ft 

-0.43 

-0.31 

-0.26 

-0.52 

JND  (%) 

49.4 

50.7 

50.2 

49.0 

MeanJND  =  49.8% 

Table  B.15:  1I-2AFC  discrimination  results  for  subject  JK  under  thimble  condition  (Standard: 
CR4  [0.293mm/N]) 


Specimen 

DR2 

DR3 

ER4 

ER3 

Difference 

3.0% 

6.7% 

12.2% 

16.2% 

Confusion 

67  29 

76  20 

83  13 

89  7 

Matrix 

12  84 

10  86 

0  96 

0  96 

d' 

1.67 

2.07 

3.67 

4.02 

/3 

-0.32 

-0.22 

-0.73 

-0.55 

JND  (%) 

1.8 

3.3 

4.0 

MeanJND  =  3.1% 

Table  B.16:  S-2AFC  discrimination  results  for  subject  JK  under  normal  condition  (Standard: 
CR3  [0.300mm/N]) 


Specimen 

DR2 

DRl 

DR4 

ER4 

Difference 

3.7% 

4.9% 

8.7% 

13.1% 

Confusion 

75  21 

77  19 

85  11 

90  6 

Matrix 

11  85 

14  82 

7  89 

2  94 

d' 

1.98 

1.90 

2.66 

3.57 

n 

-0.20 

-0.10 

-0.13 

-0.25 

JND  (%) 

1.9 

2.6 

3.3 

3.7 

MeanJND  =  2.9% 

Table  B.li :  S-2AFC  discrimination  results  for  subject  JK  under  finger  cot  condition  (Stand¬ 
ard:  CR2  [0.298mm/N]) 


Specimen 

FR6 

HT2 

IT2 

JT2 

Difference 

30.8% 

55.0% 

77.1% 

110.3% 

Confusion 

55  41 

68  28 

75  21 

78  18 

Matrix 

36  60 

31  65 

19  77 

17  79 

d' 

0.50 

1.01 

1.62 

1.81 

-0.07 

0.05 

-0.04 

-0.02 

JND  (%) 

61.4 

54.6 

47.5 

60.9 

MeanJND  =  56.1% 

Table  B.18:  S-2AFC  discrimination  results  for  subject  JK  under  thimble  condition  (Standard: 
CR6  [0.290mm/N]) 
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Appendix  C 


Statistical  Tests  on  Softness 
Discrimination  JND  Data 


The  analysis  of  variance  (ANOVA)  method  [31] [24] [36]  was  used  to  find  out  whether  the 
results  on  softness  discrimination  were  influenced  by  factors  such  as  Anger  conditions  and 
experimental  paradigms.  The  .JND  estimates  from  the  pooled  data  for  each  of  the  comparison 
specimens  were  used  for  the  calculations  of  the  ANOVA  table.  To  compare  any  two  .JND 
results  obtained  under  different  circumstances,  the  Student's  t  test  was  used.  The  statistical 
analysis  was  performed  by  using  MATLAB  statistics  toolbox. 

C.l  11-2 AFC  experiments 

A  two-way  ANOVA  was  performed  using  subjects  and  conditions  as  the  two  factors.  The 
thimble  condition  was  not  included  because  the  data  range  is  very  different  and  it  is  clear 
that  there  is  a  significant  difference  between  thimble  and  the  other  two  conditions.  Table  C.l 
lists  the  data  used  for  the  ANOVA  analysis.  The  ANOVA  table  generated  by  using  MATLAB 
software  function  anovaB  is  shown  in  Table  C.2.  The  ANOVA  analysis  provided  tests  for  the 
following  three  hypotheses.  First,  the  conditions  differ  in  performance.  Second,  the  subjects 
differ  in  performance.  Third,  there  is  an  interaction  between  the  subjects  and  conditions. 
The  ANOVA  table  provides  a  means  to  look  at  variations  between  groups  and  variations 
within  groups.  The  column  SS  in  the  table  stands  for  sum  of  squares.  The  df  column  is 
the  degrees  of  freedom  for  the  factors.  There  are  3  subjects,  therefore  the  df  for  columns 
is  2  (3  —  1).  There  are  2  conditions,  so  the  df  for  rows  is  1  (2  —  1).  The  column  MS  was 
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Condition 

Subject 

CH 

CT 

JK 

Normal 

3.8/4.6/8.1/7.8 

7.0/18.5/19.9/13.8 

2.4/7.3/6.2/12.0 

Finger  cot 

1.9/5.3/4.1/5.9 

13.1/10.4/14.8/17.7 

3.2/6.2/4.2/6.5 

Table  C.l;  1I-2AFC  data  used  for  the  two-way  ANOVA  table. 


ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

416.3 

2 

208.2 

17.92 

Ro\ws 

13.37 

1 

13.37 

1.151 

interaction 

1.558 

2 

0.7791 

0.06707 

Error 

209.1 

18 

11.62 

Total 

640.3 

23 

Table  C.2:  1I-2.4FC  ANOVA  table.  The  rows  represent  the  conditions  and  the  columns 
represent  the  subjects. 

obtained  by  dividing  SS  by  df.  The  F  ratio  was  obtained  by  dividing  the  MS  value  for 
Columns,  Rows,  and  Interaction  by  the  MS  value  for  Error.  By  looking  up  F-distribution 
table  [47]  with  degree  of  freedom  pairs  of  (2,18),  (1,18),  and  (2,18),  the  1%  points  of  F  ratios 
from  the  table  were  6.01,  8.28,  and  6.01.  Therefore,  only  the  first  hypothesis  was  rejected 
because  the  F  value  from  the  ANOVA  table  (18.46)  was  larger  than  the  F  value  (6.01)  from 
the  F  distribution.  There  was  significant  difference  between  the  subjects  at  1%  significance 
level  (the  significance  is  the  probability  of  observing  the  given  result  by  chance  given  that  the 
hypothesis  is  true).  However,  the  differences  between  normal  and  finger  cot  conditions  and 
the  interactions  of  subjects  and  conditions  were  small,  as  shown  by  the  small  F  ratio  from 
analysis  of  variance. 

A  box  plot  of  the  softness  discrimination  JND  values  obtained  under  normal  and  finger 
cot  conditions  of  the  1I-2AFC  paradigm  is  shown  in  Figure  C-1.  The  box  has  lines  at  the 
lower  quartile,  median,  and  upper  quartile  values.  The  whiskers  are  lines  extending  from 
each  end  of  the  box  to  show  the  extent  of  data. 

Three  t  tests  were  performed  on  the  data  given  in  Table  C.l  between  the  subjects’  perform¬ 
ance  under  normal  and  finger  cot  conditions  by  using  the  ttest2  function  of  the  MATLAB 
software.  Significant  differences  of  the  mean  JNDs,  at  5%  significance  level,  were  repor- 
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1I-2AFCB0X  Plot 


Figure  C-1:  A  box  plot  of  the  softness  discrimination  JND  values  obtained  under  normal(N) 
and  finger  cot(C)  conditions  of  the  1I-2AFC  paradigm. 

ted  between  subjects  (CH,CT)  under  normal  condition  and  between  subjects  (CH,CT)  and 
(JK,CT)  under  finger  cot  condition. 

C.2  S-2AFC  experiments 

The  same  type  of  analysis  used  on  the  1I-2AFC  data  was  performed  on  the  data  from  S-2AFC 
experiments.  A  two-way  ANOVA  was  performed  by  using  subjects  and  conditions  as  the  two 
factors.  The  thimble  condition  was  not  included  because  the  data  range  is  verj^  different  and 
it  is  clear  that  there  is  a  significant  difference  between  thimble  and  the  other  two  conditions. 
Table  C.3  lists  the  data  used  for  the  ANOVA  analysis.  The  ANOVA  table  generated  by  using 
MATLAB  software  function  anova2  is  shown  in  table  C.4.  There  was  significant  difference 
between  the  subjects  at  1%  significance  level  (the  significance  is  the  probability  of  observing 
the  given  result  by  chance  given  that  the  hypothesis  is  true).  However,  the  differences  between 
normal  and  finger  cot  conditions  and  the  interactions  of  subjects  and  conditions  were  small 
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Condition 

Subject 

CH 

CT 

JK 

Normal 

2.3/4.8/6.1/6.3 

8.3/6.6/8.6/9.7 

1.8/3.3/3.3/4.0 

Finger  cot 

2.4/2.9/3.7/4.0 

11.8/8.2/8.5/13.1 

1.9/2.6/3.3/3.7 

Table  C.3:  S-2AFC  data  used  for  the  two-way  ANOVA  table. 


ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

185.6 

2 

92.79 

42.95 

Rows 

0.04167 

1 

0.04167 

0.01929 

Interaction 

14.16 

2 

7.08 

3.278 

Error 

38.88 

18 

2.16 

Total 

238.7 

23 

Table  C.4:  S-2AFC  ANOVA  table.  The  rows  represent  the  conditions  and  the  columns 
represent  the  subjects. 

as  shown  by  the  small  F  ratio  from  the  analysis  of  variance. 

A  box  plot  of  the  softness  discrimination  JND  values  obtained  under  normal  and  finger 
cot  conditions  of  the  S-2AFC  paradigm  is  shown  in  Figure  C-2.  Three  t.  tests  were  performed 
on  the  data  given  in  Table  C.3  between  the  subjects  performance  under  normal  and  finger 
cot  conditions  by  using  the  ttest2  function  of  the  MATLAB  software.  Significant  differences 
of  the  mean  .JNDs,  at  5%  significance  level,  were  reported  between  subjects  (CH,CT)  and 
(JK,CT)  under  both  normal  and  finger  cot  conditions. 


C.3  Comparison  of  JND  among  different  paradigms 

To  compare  the  effect  of  the  two  experimental  paradigms,  data  from  the  three  subjects  under 
both  normal  and  finger  cot  conditions  were  used.  Pooling  data  from  all  the  subjects  together 
allows  us  to  examine  the  diiferences  between  the  two  paradigms  with  higher  reliability. 

The  ANOVA  table  generated  is  shown  in  Table  C.5.  The  two  paradigms  are  represented 
by  the  rows  in  the  table.  The  columns  represent  the  six  groups  of  data  from  the  three  subjects 
under  both  normal  and  finger  cot  conditions.  The  F  ratio  (16.21)  obtained  was  higher  than  the 
value  (7.39)  from  the  F  distribution  table  with  degrees  of  freedom  (1,36)  at  1%  significance 
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S-2AFC  Box  Plot 


Figure  C-2:  A  box  plot  of  the  softness  discrimination  JND  values  obtained  under  normal(N) 
and  finger  cot(C)  conditions  of  the  S-2AFC  paradigm. 

level.  Therefore,  we  conclude  the  results  from  the  1I-2AFC  experiments  are  significantly 
different  from  the  results  from  S-2AFC  experiments  at  1%  significance  level. 

The  individual  box  plots  that  compare  JNDs  from  the  two  paradigms  for  the  three  subjects 
are  graphed  in  Figure  C-3,  Figure  C-4,  and  Figure  C-5,  respectively.  The  results  from  the 
two  different  experimental  paradigms  are  plotted  side  by  side  for  both  normal  and  finger  cot 
conditions.  From  the  box  lines  (indicating  the  lower  quartile,  median,  and  the  upper  quartile) 
and  the  whiskers  (indicating  the  extent  of  the  data),  we  can  see  that  the  variation  in  the 
JND  values  are  lower  in  the  S-2AFC  paradigm  than  in  the  1I-2AFC  paradigm.  Therefore, 
S-2AFG  experiments  is  different  from  a  hypothetical  two  interval,  two  alternative,  forced 
choice  paradigm  (2I-2AFC)  which  would  have  higher  variations  in  the  JND  values  than  the 
1I-2AFC  experiments. 
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ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

591.5 

5 

118.3 

17.42 

Rows 

110.1 

1 

110.1 

16.21 

Interaction 

43.79 

5 

8.758 

1.29 

Error 

244.5 

36 

6.791 

Total 

989.9 

47 

Table  C.5:  ANOVA  table  to  compare  1I-2AFC  and  S-‘2AFC  paradigms.  The  rows  represent 
the  experimental  paradigms  and  the  six  columns  represent  three  subjects  under  both  normal 
and  finger  cot  conditions. 


Box  Plot:  1I-2AFC  vs.  S-2AFC 


Figure  C-3:  Box  plot  of  the  JND  values  obtained  under  both  normal(N)  and  finger  cot(C) 
conditions  using  1I-2AFC(1)  and  S-2AFC(2)  paradigms  for  subject  CH. 
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CH-N-1 


CH-N-2  CH-C-1 

Subject-Condition-Paradigm 


CH-C-2 


Figure  C-4;  Box  plot  of  the  JND  values  obtained  under  both  norrnal(N)  and  finger  cot(C) 
conditions  using  1I-2AFC(1)  and  S-2AFC(2)  paradigms  for  subject  CT. 


Box  Plot:  11-2AFC  vs.  S-2AFC 


JK-N-1  JK-N-2  JK-C-1  JK-C-2 

Subject-Condition-Paradigm 


Figure  C-5:  Box  plot  of  the  JND  values  obtained  under  both  normal(N)  and  finger  cot(C) 
conditions  using  1I-2AFC(1)  and  S-2AFC(2)  paradigms  for  subject  JK. 
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Appendix  D 


Force  Control  Data 


Presented  in  this  appendix  are  the  data  recorded  during  the  force  control  experiments  under 
various  conditions.  For  each  subject,  three  trials  of  force  data  under  the  same  conditions 
are  plotted  on  three  separate  pages.  There  are  four  rows  to  each  graph  which  represent  the 
specimens  used.  Going  from  top  to  bottom  are  rigid,  hard,  medium,  and  soft  specimens. 
The  Ple.xiglas  was  used  as  the  rigid  specimen  and  the  three  compliant  specimens  chosen  had 
objective  compliances  of0..304mm/N  (hard),  0.455mm/N  (medium),  and  0.608mm/N  (soft). 
The  three  columns  represent  the  data  from  normal,  finger  cot,  and  thimble  conditions.  The 
subject,  hand,  and  trial  number  are  printed  on  the  upper-left  corner  of  the  force  versus  time 
graphs. 
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5  10 

Finger  cot 


15 


5  10 

Thimble 


Soft(JT3)  Medium  (HT1)  Hard(CRI)  Rigid 


Subject:CH  Hand:R  Trial:3 
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Soft(JT3)  Medium  (HT1)  Hard(CR1)  Rigid 


8 


SubjectCH  Hand:L  Trial;  1 
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15 


15 


15 


5  10 

Normal 


5  10 

Finger  cot 


5  10 

Thimble 
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Medium  (HT1) 


15 


5  10 

Thimble 


Soft  (JT3) 


Normal 


Finger  cot 


Thimble 
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Soft(JT3)  Medium  (HT1)  Hard(CR1)  Rigid 


Subject;CT  Hand;R  Trial:3 
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5  10  15  5  10  15  5  10  15 

Normal  Finger  cot  Thimble 
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148 


Normal 


Finger  cot 


Thimble 
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5  10  15  5  10  15  5  10  15 

Normal  Finger  cot  Thimble 
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Soft(JT3)  Medium  (HT1)  Hard  (CR1)  Rigid 


5  10  15  5  10  15 

Normal  Finger  cot 
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Soft(JT3)  Medium  (HT1)  Hard(CRI)  Rigid 


Subject:JK  Hand:L  Trial;1 


-/VN/^k  . 

1  Am  /n^ 

5  10  15 


152 


15 


15 


5  10 

Normal 


5  10 

Finger  cot 
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5  10 

Finger  cot 


15 
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Appendix  E 


Force  Control  Performance  Measures 


The  force  data  digitized  at  200  samples  per  second  were  filtered  with  a  low-pass  filter  which 
had  a  cut-off  frequency  of  50Hz  for  noise  removal  before  the  force  error  was  calculated.  Also, 
onlj'  the  data  from  the  second  and  third  stage  of  each  run  were  analyzed.  During  the  second 
stage,  the  subject  had  visual  force  feedback;  during  the  third  stage,  the  visual  feedback  was 
turned  off.  The  mean  absolute  error  and  the  pooled  standard  deviation  were  tabulated. 
The  mean  absolute  error  was  obtained  from  3  experimental  runs  of  1000  data  points  during 
each  stage.  The  standard  deviation  (s)  was  obtained  by  pooling  the  standard  deviations 
from  the  three  experimental  runs  (si,  S2,  and  S3)  according  to  the  following  formula:  s  = 
y(sp  +  S2^  +  S3^)/3. 
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Condition 


Normal 


Normal 


Normal 


Finger  cot 


Finger  cot 


Finger  cot 


Thimble 


Thimble 


Thimble 


Rigid 


Mean  S.D. 


0.162 


0.093  I  0.050 


.586  0.301 

.148  0.144 


(U71  I  0.098 
0.5 


3 


Absolute 


Hard 

Mean  |  S.D. 


0.107 


0.314 


0.133 


0.218 


0.096 


0.067 


0.267 


0.105  0.054 


0.533 


0.266  I  0.120 


0.090  0.041 


0.264 


Medium 

Mean 

S.D. 

0.150 

0.101 

0.107 

0.059 

0.331 

0.182 

0.111 

0.083 

0.062 

0.038 

0.400 

0.247 

0.082 

0.051 

0.195 

0.121 

0.506 

0.336 

Soft 


Mean  S.D. 


0.123 


0.135 


0.164 


0.157 


0.098 


0.214  0.091 


0.104  0.095 


0.415  0.321 


Table  E.2:  Absolute  error  in  force  control  for  subject  CH  using  right  hand  index  finger  without 
visual  feedback. 


Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N 

0.037 

0.026 

0.033 

0.026 

0.035 

0.025 

0.038 

0.031 

Normal 

4N 

0.054 

0.045 

0.048 

0.044 

0.047 

0.039 

0.069 

0.077 

Normal 

8N 

0.125 

0.104 

0.086 

0.079 

0.106 

0.083 

0.107 

0.117 

Finger  cot 

2N 

0.048 

0.039 

0.043 

0.039 

0.033 

0.030 

0.041 

0.033 

Finger  cot 

4N 

0.062 

0.058 

0.065 

0.040 

0.056 

0.043 

0.071 

0.075 

Finger  cot 

8N 

0.136 

0.095 

0.069 

0.058 

0.100 

0.113 

0.081 

0.069 

Thimble 

2N 

0.040 

0.041 

0.029 

0.025 

0.041 

0.034 

0.048 

0.027 

Thimble 

4N 

0.074 

0.052 

0.037 

0.035 

0.065 

0.044 

0.071 

0.047 

Thimble 

8N 

0.110 

0.080 

0.076 

0.057 

0.068 

0.052 

0.108 

0.090 

Table  E.3:  Absolute  error  in  force  control  for  subject  CH  using  left  hand  index  finger  with 
visual  feedback. 


Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N 

0.188 

0.076 

ES^I 

0.056 

0.168 

0.083 

Normal 

4N 

0.252 

0.144 

130^ 

0.060 

0.056 

0.111 

0.073 

Normal 

8N 

0.218 

0.174 

0.129 

0.101 

Finger  cot 

2N 

0.324 

0.160 

0.154 

112013 

0.210 

0.135 

Finger  cot 

4N 

0.204 

0.102 

0.263 

0.119 

0.176 

0.074 

0.345 

0.162 

Finger  cot 

8N 

0.172 

0.329 

0.286 

0.335 

0.256 

0.150 

Thimble 

liWEa 

0.051 

0.044 

0.316 

0.158 

Thimble 

4N 

0.280 

0.124 

0.092 

0.057 

0.326 

0.103 

Thimble 

8N 

0.180 

0.121 

0.168 

0.132 

0.351 

0.171 

Table  E.4:  Absolute  error  in  force  control  for  subject  CH  using  left  hand  index  finger  without 
visual  feedback. 
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Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N 

giinigi 

0.043 

0.021 

0.016 

Normal 

4N 

0.066 

0.048 

0.080 

0.060 

0.079 

0.058 

liliSSl 

0.037 

Normal 

8N 

0.147 

0.102 

0.062 

lilHI 

0.098 

Finger  cot 

2N 

0.032 

0.019 

Finger  cot 

4N 

0.071 

0.053 

Finger  cot 

8N 

0.188 

0.051 

0.114 

Thimble 

2N 

0.041 

0.032 

0.027 

0.019 

0.028 

0.024 

0.025 

0.016 

Thimble 

4N 

0.074 

0.065 

0.082 

0.072 

0.048 

0.033 

0.042 

0.024 

Thimble 

8N 

0.142 

0.104 

0.139 

0.152 

0.136 

0.093 

0.137 

0.080 

Table  E.5:  Absolute  error  in  force  control  for  subject  CT  using  right  hand  index  finger  with 
visual  feedback. 


Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N 

0.157 

0.103 

0.179 

0.105 

0.079 

0.073 

0.184 

0.139 

Normal 

4N 

0.565 

0.275 

0.308 

0.140 

0.311 

0.160 

0.509 

0.263 

Normal 

8N 

1.400 

0.849 

0.931 

0.436 

0.609 

0.371 

0.875 

0.461 

Finger  cot 

2N 

0.130 

0.075 

0.082 

0.098 

0.068 

0.202 

0.113 

Finger  cot 

4N 

0.646 

0.338 

0.507 

0.207 

0.568 

0.304 

0.437 

0.224 

Finger  cot 

8N 

1.303 

0.560 

0.833 

0.441 

0.957 

0.520 

0.642 

0.331 

Thimble 

2N 

0.101 

0.058 

0.100 

0.054 

0.060 

0.038 

0.127 

0.083 

Thimble 

4N 

0.156 

0.110 

0.541 

0.240 

0.221 

0.135 

0.532 

0.335 

Thimble 

8N 

1.137 

0.568 

0.742 

0.360 

1.156 

0.660 

0.833 

0.406 

Table  E.6:  Absolute  error  in  force  control  for  subject  CT  using  right  hand  index  finger  without 
visual  feedback. 
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Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

■2N 

0.029 

0.020 

0.032 

0.030 

0.035 

0.023 

0.040 

0.044 

Normal 

4N 

0.051 

0.042 

0.066 

0.057 

0.045 

0.034 

0.054 

0.034 

Normal 

8N 

0.132 

0.080 

0.133 

0.083 

0.106 

0.077 

0.145 

0.086 

Finger  cot 

2N 

0.045 

0.051 

0.034 

0.034 

0.053 

0.076 

0.036 

0.038 

Finger  cot 

4N 

0.048 

0.036 

0.065 

0.039 

0.063 

0.054 

0.057 

0.053 

Finger  cot 

8N 

0.154 

0.083 

0.114 

0.087 

0.105 

0.071 

0.113 

0.075 

Thimble 

2N 

0.044 

0.046 

0.038 

0.034 

0.042 

0.040 

0.034 

0.033 

Thimble 

4N 

0.069 

0.058 

0.056 

0.043 

0.048 

0.029 

0.053 

0.041 

Thimble 

8N 

0.162 

0.094 

0.132 

0.081 

0.117 

0.073 

0.105 

0.063 

Table  E.7:  AI)solute  error  in  force  control  for  subject  CT  using  left  hand  index  finger  with 
visual  feedback. 


Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft  1 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N 

0.055 

0.046 

0.053 

0.033 

0.087 

0.062 

0.074 

0.044 

Normal 

4N 

0.206 

0.126 

0.283 

0.127 

0.267 

0.157 

0.320 

0.160 

Normal 

8N 

0.821 

0.375 

0.537 

0.283 

0.830 

0.432 

1.011 

0.598 

Finger  cot 

2N 

0.104 

0.040 

0.036 

0.023 

0.073 

0.047 

0.057 

0.033 

Finger  cot 

4N 

0.149 

0.102 

0.155 

0.139 

0.207 

0.141 

0.207 

0.144 

Finger  cot 

8N 

0.902 

0.565 

1.111 

0.562 

0.896 

0.530 

0.682 

0.380 

Thimble 

2N 

0.051 

0.031 

0.063 

0.031 

0.096 

0.080 

0.033 

0.024 

Thimble 

4N 

0.166 

0.103 

0.262 

0.176 

0.158 

0.095 

0.193 

0.137 

Thimble 

8N 

0.998 

0.531 

0.878 

0.519 

0.872 

0.557 

0.607 

0.407 

Table  E.8:  Absolute  error  in  force  control  for  subject  CT  using  left  hand  index  finger  without 
visual  feedback. 
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Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

2N 

0.033 

■IMIIBI 

0.037 

0.028 

0.029 

0.023 

Normal 

4N 

KifiklJ 

0.063 

0.050 

0.058 

Normal 

8N 

QQI 

0.114 

0.090 

0.151 

0.108 

0.134 

Finger  cot 

2N 

liMikea 

0.027 

0.022 

0.046 

0.037 

0.034 

0.030 

0.059 

0.036 

0.068 

0.063 

0.066 

itftbgl 

0.121 

0.099 

0.089 

0.139 

■imiEB 

Thimble 

hS2!£fi| 

0.026 

IQSgQI 

0.048 

0.047 

Thimble 

4N 

■IMiKM 

0.032 

mm 

Thimble 

8N 

kiMIKW 

0.073 

liUIi 

Table  E.9;  Absolute  error  in  force  control  for  subject  JK  using  right  hand  index  finger  with 
visual  feedback. 


Condition 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

S.D. 

Mean 

Mean 

S.D. 

Normal 

BSi 

0.114 

0.074 

0.087 

0.080 

0.084 

0.054 

0.065 

Normal 

BSfl 

EBI 

IHIqQ 

0.221 

0.087 

0.123 

iilikfci 

Normal 

8N 

0.311 

0.212 

0.217 

0.148 

0.475 

0.260 

Finger  cot 

0.048 

0.173 

0.160 

0.098 

0.070 

0.117 

0.098 

Finger  cot 

4N 

0.109 

0.081 

0.125 

0.087 

0.186 

0.144 

0.204 

0.132 

Finger  cot 

8N 

0.513 

0.292 

0.330 

0.187 

0.289 

0.177 

0.306 

0.184 

Thimble 

0.104 

0.078 

0.046 

0.128 

0.068 

0.125 

0.121 

Thimble 

4N 

0.158 

0.106 

0.105 

0.067 

0.156 

0.117 

0.094 

0.062 

Thimble 

8N 

0.299 

0.207 

0.291 

0.149 

0.213 

0.156 

0.455 

0.368 

Table  E.IO:  Absolute  error  in  force  control  for  subject  JK  using  right  hand  index  finger 
without  visual  feedback. 
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Condition 

Force 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Mean 

S.D. 

Normal 

KASfiGi 

ESSO 

Normal 

4N 

Hilyglgl 

mmm 

0.089 

0.090 

EESd 

Normal 

8N 

BKHil 

0.158 

0.1.52 

Finger  cot 

2N 

lililW 

|l]Q^ 

0.044 

0.036 

0.049 

0.033 

0.059 

0.042 

Finger  cot 

4N 

0.069 

0.052 

0.064 

Finger  cot 

8N 

0.140 

giWBil 

0.141 

0.131 

Thimble 

2N 

0.065 

0.046 

0.060 

0.039 

0.044 

0.033 

Thimble 

4N 

0.064 

0.048 

0.061 

0.043 

0.071 

0.0.54 

Thimble 

0.115 

0.097 

0.1.50 

0.125 

Table  E.ll;  Absolute  error  in  force  control  for  subject  JK  using  left  hand  index  finger  with 
visual  feedback. 


Condition 

Absolute  Error  (N) 

Rigid 

Hard 

Medium 

Soft 

Mean 

S.D. 

S.D. 

S.D. 

Normal 

0.101 

0.084 

0.088 

0.072 

ES^ 

0.199 

0.165 

Normal 

0.155 

0.174 

0.171 

0.206 

ESE3 

0.389 

0.255 

Normal 

8N 

0.511 

0.575 

iillfefcfcli 

Finger  cot 

2N 

0.145 

0.097 

0.138 

0.104 

0.085 

0.059 

0.169 

o.iiq 

Finger  cot 

4N 

0.266 

0.187 

0.307 

0.229 

0.208 

0.159 

Finger  cot 

8N 

1-.016 

0.536 

0.585 

0.341 

0.648 

0.399^ 

1.041 

0.613 

Thimble 

EE9I 

0.198 

0.145 

0.130 

0.072 

0.071 

0.048 

Thimble 

EESI 

0.227 

0.141 

0.132 

0.080 

0.156 

0.248 

Thimble 

EESI 

0.416 

0.331 

0.489 

0.393 

0.777 

0.533 

1.009 

0.523 

Table  E.12;  Absolute  error  in  force  control  for  subject  JK  using  left  hand  index  finger  without 
visual  feedback. 
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Appendix  F 


Statistical  Tests  on  Force  Control 
Performance  Measures 


The  aiiafysis  of  variance  (ANOYA)  method  was  used  to  find  out  whether  the  force  control 
performance  was  influenced  b_y  factors  such  as  target  force  magnitude,  finger  contact  con¬ 
ditions,  and  specimens.  The  performance  data  from  Appendix  E  were  used  for  all  the  tests 
performed.  All  the  statistical  analyses  were  performed  by  using  MATLAB  statistics  toolbox. 

F.l  Specimen  softness  and  finger  contact  conditions 

The  effects  of  both  specimen  softness  and  finger  contact  conditions  were  examined  together 
by  using  two-way  ANOVA.  The  data  from  the  three  subjects  for  the  two  visual  force  feedback 
conditions  and  two  hands  were  analyzed  separately  (a  total  of  12  combinations).  In  other 
words,  the  data  for  the  three  target  force  magnitudes  were  grouped  together  in  this  analysis. 
Visual  observations  of  the  subjects’  performance  indicated  that  they  seemed  to  have  higher 
error  at  high  target  force  magnitude.  To  minimize  the  effect  of  target  force  in  the  analysis, 
the  performance  data  was  expressed  in  terms  of  percentage  of  the  target. 

The  results  of  the  analysis  could  not  reject  the  hypothesis  that  there  was  no  effects  of 
specimen  softness  and  of  the  contact  conditions  in  all  but  one  circumstances  (1  out  of  12  tests) . 
In  that  only  circumstance,  the  ANOVA  resulted  in  rejecting  the  hypothesis  of  no  specimen 
effect  for  subject  JK  under  visual  force  feedback.  Based  on  results  of  the  majority  of  the 
conditions  tested,  the  likelihood  that  the  specimen  softness  and  the  finger  contact  conditions 
played  important  roles  in  the  force  control  tasks  was  small.  Therefore,  the  following  analysis 
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used  data  pooled  from  the  different  specimen  softnesses  as  well  as  various  contact  interface 
conditions. 

F.2  Target  force  magnitude 

One-way  ANOVA  tables  were  constructed  to  compare  target  force  magnitude  factor.  The 
columns  stands  for  the  data  collected  at  three  target  force  levels  under  various  specimens 
and  contact  interface  conditions.  The  reason  for  pooling  data,  from  those  two  factors  is  to 
increase  the  reliability  of  the  analysis.  Table  F.l  and  Table  F.2  list  the  data  pooled  for  the 
two  visual  force  feedback  conditions.  The  resulting  ANOVA  tables  are  shown  in  Tables  F.3 
and  Table  F.4.  The  hypothesis  tested  here  is  that  target  force  magnitude  did  not  significantly 
affect  the  results  in  either  conditions.  Taking  the  F  ratios  obtained  from  the  two  ANOVA 
tables  (32.06  and  36.59)  and  comparing  them  to  the  F  distribution  with  the  same  degree  of 
freedoms  (2,33),  both  hypotheses  were  rejected  at  1  %  significance  level.  The  target  force 
magnitude  was  found  to  be  a  significant  factor  that  affected  control  performance.  However, 
when  only  comparing  data  from  2N  and  4N  target  force,  the  hypothesis  could  not  be  rejected 
under  both  visual  force  feedback  conditions.  Thus,  the  main  differences  came  from  the  8N 
target  force  runs  for  subject  CH. 

The  corresponding  box  plots  which  show  the  spread  of  the  data  for  the  tw'o  tables  are 
shown  in  Figure  F-1  and  Figure  F-2.  The  box  has  lines  at  the  lower  quartile,  median,  and 
upper  quartile  values.  The  whiskers  are  lines  extending  from  each  end  of  the  box  to  show  the 
extent  of  the  rest  of  the  data  if  they  were  less  than  1.5  times  the  interquartile  ranges  (IQR). 
Otherwise,  whiskers  end  at  the  most  extreme  observations,  still  lie  within  1.5  times  IQR  of 
the  quartiles  and  the  remaining  observations  were  plotted  as  outliers. 

For  subjects  CT  and  JK,  there  were  also  significant  differences  between  2N,  4N,  and 
8N  target  force  magnitudes.  In  addition,  the  differences  between  2N  and  4N  target  force 
magnitudes  were  big  enough  to  reject  the  hypothesis  that  their  performance  were  the  same 
at  1%  significance  level  with  and  without  visual  force  feedback.  The  box  plots  for  the  two 
subjects  under  those  two  conditions  are  shown  in  Figure  F-3,  Figure  F-4,  and  Figure  F-5, 
Figure  F-6,  respectively. 

On  the  other  hand,  comparison  can  be  made  after  the  absolute  errors  were  divided  by  the 
target  force  magnitudes  and  the  performance  is  measured  in  terms  of  the  percentage  error. 
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Specimen 

Finger 

Condition 

Mean  Absolute  Error  (N) 

Target:  4N 

Target:  8N 

Rigid 

Normal 

0.035 

0.054 

Finger  cot 

0.051 

0.070 

Thimble 

0.048 

Hard 

Normal 

Hard 

Finger  cot 

0.037 

Hard 

Thimble 

0.057 

0.032 

0.098 

Medium 

Normal 

0.030 

0.056 

0.128 

Medium 

Finger  cot 

0.032 

0.049 

0.108 

Medium 

Thimble 

0.030 

0.060 

0.066 

Soft 

Normal 

0.036 

0.067 

0.073 

Soft 

Finger  cot 

0.035 

0.0.57 

0.160 

Soft 

Thimble 

0.041 

0.042 

0.110 

Table  F.l:  Data  used  for  computing  a  one-way  ANOVA  table  to  determine  the  significance 
of  target  force  magnitude  for  subject  CH  with  visual  feedback  when  using  the  right  hand. 


For  both  subjects  CH  and  .JK,  the  percentage  errors  were  significantly  lower  when  controlling 
4N  and  8N  target  force  than  when  controlling  2N  target  force  with  visual  force  feedback  (see 
Figure  F-7  and  Figure  F-9).  For  subject  CT,  the  percentage  errors  were  not  significantly 
different  under  the  three  different  target  force  levels  with  visual  feedback  (see  Figure  F-8). 
The  comparisons  for  the  performance  achieved  at  different  target  force  magnitude  without 
visual  feedback  were  conducted  in  a  similar  fashion.  The  results  are  shown  in  Figure  F-10, 
Figure  F-11,  and  Figure  F-12. 

The  qualitative  effect  of  target  force  magnitude  as  observed  from  the  box  plots  can  be 
summarized  as  follows:  When  the  mean  absolute  error  is  used  as  the  performance  index, 
higher  target  force  generally  increases  both  the  mean  absolute  error  and  the  standard  deviation 
of  the  absolute  error.  However,  for  subject  CH,  the  performance  at  2N  and  4N  with  or  without 
visual  feedback  were  about  the  same.  When  the  mean  absolute  error  expressed  in  percentage 
is  used  as  the  performance  index,  higher  target  force  did  not  increase  the  error  in  general, 
except  for  subject  CT  without  visual  force  feedback. 

F.3  Visual  feedback 

One-way  ANOVA  tables  were  constructed  using  the  performance  data,  expressed  in  percent¬ 
age  of  target  force,  collected  from  each  subject,  to  compare  the  effect  of  visual  feedback 


164 


Specimen 

Finger 

Condition 

Mean  Absolute  Error  (N) 

Target:  2N 

Target:  4N 

Target:  8N 

Rigid 

Normal 

0.162 

0.093 

0.586 

Rigid 

Finger  cot 

0.148 

0.171 

0.519 

Rigid 

Thimble 

0.121 

0.105 

0.533 

Hard 

Normal 

0.191 

0.261 

0.210 

Hard 

Finger  cot 

0.182 

0.107 

0.314 

Hard 

Thimble 

0.266 

0.090 

0.520 

Medium 

Normal 

0.150 

0.107 

0.331 

Medium 

Finger  cot 

0.111 

0.062 

0.400 

Medium 

Thimble 

0.082 

0.195 

0.506 

Soft 

Normal 

0.123 

0.135 

0.164 

Soft 

Finger  cot 

0.157 

0.098 

0.594 

Soft 

Thimble 

0.214 

0.104 

0.415 

Table  F.2;  Data  used  for  computing  one-\va,v  ANOVA  table  to  determine  the  significance  of 
target  force  magnitude  for  subject  CH  without  visual  feedback  when  using  the  right  hand. 


ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

0.03064 

2 

0.01532 

32.06 

Error 

0.01577 

33 

0.0004778 

Total 

0.04641 

35 

Table  F.3:  ANOVA  table  to  determine  the  significance  of  target  force  magnitude  for  subject 
Cn  with  visual  feedback  when  using  right  hand.  The  columns  represent  the  data  obtained 
from  three  target  forces  under  various  finger  and  specimen  conditions. 


ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

0.6386 

2 

0.3193 

36.59 

Error 

0.288 

33 

0.008727 

Total 

0.9266 

35 

Table  F.4;  ANOVA  table  to  determine  the  significance  of  target  force  magnitude  for  subject 
CH  without  visual  feedback  when  using  right  hand.  The  columns  represent  the  data  obtained 
from  three  target  forces  under  various  finger  and  specimen  conditions. 
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Subject  CH  with  Visual  Feedback 


Figure  F-1:  Box  plot  of  data,  pooled  from  various  finger  and  specimen  conditions  for  subject 
CH  with  visual  feedback  when  using  the  right  hand. 


Subject  CH  without  Visual  Feedback 


Figure  F-2:  Box  plot  of  data  pooled  from  various  finger  and  specimen  conditions  for  subject 
CH  without  visual  feedback  when  using  the  right  hand. 
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Subject  CT  with  Visual  Feedback 


Figure  F-3:  Box  plot  of  data  pooled  from  various  finger  and  specimen  conditions  for  subject 
CT  with  visual  feedback  when  using  the  right  hand. 


Subject  CT  without  Visual  Feedback 


Figure  F-4:  Box  plot  of  data  pooled  from  various  finger  and  specimen  conditions  for  subject 
CT  without  visual  feedback  when  using  the  right  hand. 
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Target  Force 


Figure  F-5:  Box  plot  of  data  pooled  from  various  finger  and  specimen  conditions  for  subject 
JK  with  visual  feedback  when  using  the  right  hand. 


Subject  JK  without  Visual  Feedback 


Figure  F-6:  Box  plot  of  data  pooled  from  various  finger  and  specimen  conditions  for  subject 
JK  without  visual  feedback  when  using  the  right  hand. 


Subject  CH  with  Visual  Feedback 


2N 


4N 

Target  Force 


8N 


Figure  F-7:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  finger  and  speci¬ 
men  conditions  for  subject  CH  with  visual  feedback  when  using  the  right  hand. 


Subject  CT  with  Visual  Feedback 


Figure  F-8;  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  finger  and  speci¬ 
men  conditions  for  subject  CT  with  visual  feedback  when  using  the  right  hand. 
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Subject  JK  with  Visual  Feedback 


4N 

Target  Force 


Figure  F-10:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  finger  and 
specimen  conditions  for  subject  CH  without  visual  feedback  when  using  the  right  hand. 
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Subject  CT  without  Visual  Feedback 


Figure  F-11:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  finger  and 
specimen  conditions  for  subject  CT  without  visual  feedback  when  using  the  right  hand. 


Subject  JK  without  Visual  Feedback 


Figure  F-12:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  finger  and 
specimen  conditions  for  subject  JK  without  visual  feedback  when  using  the  right  hand. 
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Subject  CH  with  Right  Hand 


Figure  F-13:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  CH  when  using  the  right  hand. 

by  pooling  data  from  various  finger  conditions,  target  forces,  and  specimen  softnesses.  The 
columns  stand  for  the  data  collected  with  different  force  feedback  conditions.  The  hypothesis 
tested  here  is  that  the  presence  or  absence  of  feedback  did  not  significantly  affect  the  results. 
The  ANOVA  results  rejected  the  hypothesis  for  all  three  subjects  with  very  high  F  ratios 
at  1%  significance  level.  Therefore,  there  is  a  significant  difference  between  whether  or  not 
visual  force  feedback  was  available.  Figure  F-13,  Figure  F-14,  and  Figure  F-15  show  the  box 
plots  for  the  right  hand  of  the  three  subjects. 

F.4  Hand  used 

One-way  ANOVA  tables  were  constructed  using  the  performance  data,  expressed  in  percent¬ 
age  of  target  force,  collected  from  each  subject  to  compare  the  effect  of  visual  feedback  by 
pooling  data  from  various  finger  conditions,  target  forces,  and  specimen  softness  with  visual 
feedback.  The  columns  stand  for  the  data  collected  with  different  hands.  The  hypothesis 
tested  here  is  that  the  two  hands  did  not  significantly  affect  the  results.  The  resulting  AN¬ 
OVA  tables  which  reject  the  hypothesis  at  1%  significance  level  are  listed  in  Tables  F.5,  F.6, 
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Visual  Feedback  Availability 


Figure  F-14:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  CT  when  using  the  right  hand. 


Subject  JK  with  Right  Hand 


Figure  F-1.5;  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  JK  when  using  the  right  hand. 
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ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

199.9 

1 

199.9 

14.3 

Error 

978.6 

70 

13.98 

Total 

1179 

71 

Table  F.5:  ANOVA  table  to  determine  the  significance  of  the  hand  used  for  subject  CT 
without  visual  feedback.  The  columns  represent  data,  in  percentage,  obtained  from  various 
target  forces  under  various  finger  and  specimen  conditions. 

ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

3.369 

1 

3.369 

20.17 

Error 

11.69 

70 

0.167 

Total 

15.06 

71 

Table  F.6:  ANOVA  table  to  determine  the  significance  of  the  hand  used  for  subject  JK  with 
visual  feedback.  The  columns  represent  data,  in  percentage,  obtained  from  various  target 
forces  under  various  finger  and  specimen  conditions. 

and  F.7.  The  tables  correspond  to  the  data  of  subject  CT  without  visual  feedback  and 
subject  JK  with  and  without  visual  feedback. 

Therefore,  there  was  a  significant  difference  between  the  performance  of  the  two  hands 
of  subject  JK  in  force  control  with  and  without  visual  feedback.  For  subject  CT,  there  was 
a  difference  v,fhen  the  visual  feedback  was  not  available.  As  far  as  subject  CH  is  concerned, 
the  performance  was  not  significantly  different  for  the  two  hands.  Figure  F-16  to  Figure  F- 
21  show  the  box  plots  for  the  three  subjects  using  different  hands  with  and  without  visual 
feedback. 
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ANOVA  Table 


Source 

SS 

df 

MS 

F 

Columns 

162.5 

1 

162.5 

33.77 

Error 

336.9 

70 

4.813 

Total 

499.4 

71 

Table  F.7:  ANOVA  table  to  determine  the  significance  of  the  hand  used  for  subject  JK 
without  visual  feedback.  The  columns  represent  data,  in  percentage,  obtained  from  various 
target  forces  under  various  finger  and  specimen  conditions. 


Subject  CH  with  Visual  Feedback 


Figure  F-16:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  CH  when  using  the  right  hand. 
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Subject  CH  without  Visual  Feedback 


Left  Right 

Hand  Used 

Figure  F-17;  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  CH  when  using  the  left  hand. 


Subject  CT  with  Visual  Feedback 


Figure  F-18:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  CT  when  using  the  right  hand. 
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Hand  Used 


Figure  F-19:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  CT  when  using  the  left  hand. 


Subject  JK  with  Visual  Feedback 


Figure  F-20:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force 
and  specimen  conditions  for  subject  JK  when  using  the  right  hand. 


Subject  JK  without  Visual  Feedback 


Figure  F-21:  Box  plot  of  data,  expressed  in  percentage,  pooled  from  various  interface,  force, 
and  specimen  conditions  for  subject  JK  when  using  the  right  hand. 
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Appendix  G 


Contact  Visualization  Analysis 


Presented  in  this  appendix  are  the  data  analyzed  during  the  contact  visualization  experiments 
under  various  conditions.  For  the  low  magnification  experiments  on  overall  contact  regions, 
the  results  are  plotted  in  two  ways.  In  the  first  figure,  the  various  pieces  of  contact  information 
are  plotted  against  time;  in  the  second,  the  same  information  is  plotted  against  force.  The 
video  and  force  data  were  collected  at  a  rate  of  20  and  1000  samples  per  seconds,  respectively. 
In  the  figures,  the  mean  force  during  each  frame  was  used  as  the  force  magnitude. 
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Subjectxh  Specimen:rig  Trial:  1 


Figure  G-1:  Various  contact  variables,  plotted  against  time,  calculated  from  the  image  and 
force  data  obtained  using  low  magnification  (Subject:CH,  Trial:  1). 
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Subjectxh  Specimen:rig  Trial;1 


Force  (N) 


Figure  G-2;  Various  contact  variables,  plotted  against  contact  force,  calculated  from  the 
image  and  force  data  obtained  using  low  magnification  (Subject:CH,  Triabl). 
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Figure  G-4:  Various  contact  variables,  plotted  against  contact  force,  calculated  from 
image  and  force  data  obtained  using  low  magnification  (Subject;CH,  Trial:2). 
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Figure  G-6:  Various  contact  variables,  plotted  against  contact  force,  calculated  from 
image  and  force  data  obtained  using  low  magnification  (Subject:CH,  Trial:3). 
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Figure  G-9:  Various  contact  variables,  plotted  against  time,  calculated  from  the  image  and 
force  data  obtained  using  low  magnification  (Subject;CT,  Trial:2). 
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Figure  G-10;  Various  contact  variables,  plotted  against  contact  force,  calculated  from  the 
image  and  force  data  obtained  using  low  magnification  (Subject;CT,  Trial:2). 
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Figure  G-12:  Various  contact  variables,  plotted  against  contact  force,  calculated  from  the 
image  and  force  data  obtained  using  low  magnification  (SubjectiCT,  Trial;3). 
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Figure  G-14:  Various  contact  variables,  plotted  against  contact  force,  calculated  from  the 
image  and  force  data  obtained  using  low  magnification  (Subject:JK,  Triahl). 
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Figure  G-15:  Various  contact  variables,  plotted  against  time,  calculated  from  the  image  and 
force  data  obtained  using  low  magnification  (Subject:JK,  Trial;2). 
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Figure  G-18:  Various  contact  variables, 
image  and  force  data  obtained  using  low 
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Figure  G-19:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (Subject;CH,  Specimen;  rigid). 
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Figure  G-20:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (SubjecttCH,  Specimemhard). 
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Figure  G-21:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (SubjectiCH,  Specimenmiedium). 
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Figure  G-22:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (SubjectiCH,  Specimen:soft). 
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Figure  G-23:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (SubjectiCT,  Specimen;  rigid). 
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Figure  G-24;  Percentage  contax;t  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (Subject;CT,  Specimenihard). 
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Figure  G-25:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (Subject:CT,  Specimen:medium). 
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Figure  G-26;  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (SubjectiCT,  Specimen :soft). 
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Figure  G-27:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (Subject:JK,  Specimen;  rigid). 
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Figure  G-28:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (SubjecttJK,  Specimen :hard). 
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Figure  G-29:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (Subject:JK,  Specimenmiedium). 
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Figure  G-30:  Percentage  contact  area  calculated  from  the  image  and  force  data  obtained 
using  high  magnification  (SubjecttJK,  Specimen;soft). 
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